National Commission for Academic Accreditation and Assessment PLANNING AND IMPLEMENTATION OF SELF-STUDY for Institutional Accreditation Dr. George Peterson

National Commission for Academic Accreditation

and Assessment

PLANNING AND IMPLEMENTATION OF SELF-STUDY

for Institutional Accreditation

Dr. George PetersonDr. Eqbal Darandari

May. 21-22, [email protected] © for materials are reserved for NCAAA

AIMS OF the WORKSHOP

Providing• Understanding of the process of Self Study • Guidance on the content of a Self Study Report

(SSR) and its role in the process of External Review/Accreditation– Meeting generally accepted standards in higher

education.– Institutional mission and objectives.– Research report on the quality of the institution.– Self assessment against given criteria

•

Copyrights © for materials are reserved for NCAAA

Learning Outcomes

Discuss and explain how to plan, manage, and implement

Institutional Self Study

Implement Skills of Institutional

Self Study using NCAAA templates

Lead the design and writing of

Institutional Self Study Report.

By the end of the workshop, the participant will be able to:


Day 1S 3 S 2 S 1

Using NCAAA standards and

evidencePlanning the SSR

Introduction and Purpose of the SSR

Day 2

S 3 S 2 S 1

Applications, Questions, and

Sharing Experiences

Recommendations, Action Plans and

Conclusions

Structure and Content of the SSR


Day 1S 3 S 2 S 1


evidencePlanning the SSR



Introduction

Participants Preparing for accreditationParticipants Preparing for developmental

reviews Participants about to start institutional self

study


Introduction: Important Facts

• In the last 6 years, NCAAA and its

partners made a lot of improvements. • More than 6000 faculty members have

participated in the NCAAA's local and broad training programs.

• 90% of the institutions have establish their own quality units or centres.

• Over than 70% of HEIs have made initial self evaluations at institution and program levels

• Many institutions have started a plan-implement-monitor-review cycle.


• Many institutions conducted strategic plans for quality improvement and included actions to implement the requirements of the quality assurance system dealing with significant deficiencies discovered.

• 5 institutions and 41 programs have undergone the developmental reviews.

• 6 institutions and 32 programs have been reviewed for accreditation

Introduction: Important Facts


Evaluation and Accreditation Steps

1. Establish Quality Assurance System2. Meet Institution Goals and NCAAA

Standards3. Meet Eligibility Requirements4. Prepare Self Study Report (SSR)5. External Review and Report6. Accreditation


NCAAA Accreditation Flowchart

Institution orProgram

with proper government approvals

Applicationto start

accreditation process

On-site ASG inspection,

meeting

Liaison Officer assigned by ASG

InstitutionProgram

Preparationto meet NCAAA

requirements

Evaluation date set

Liaison Officer works withInstitutionProgram Quality

Assurance Center

EligibilityRequirements

Met

Complete

Scales Report

Self StudyStrategic PlanSpecificationsKPIs in placeStandards MetShow Evidence

Accredited Institution/Program

Annual ReportsAnnual Review

· Inspections · Monitoring Strategic Plan ResultsTraining

NCAAA Secretary General approves sending Review

Panel

Recommends Review

Panel vi

NCAAA grants accreditation

RecommendsAccreditation

Every 5 years -new Self–Study and on-site Review Panel

Evaluation

On-site visit Review Panel

Evaluation and Report

RecommendsCorrections

NCAAA Secretary General approves to start

accreditation process

Accreditation Time Line

RecommendsCorrections

NCAAA Board

warns orrevokes

NCAAA Applicant Preparation to be Accredited NCAAA Accreditation

3 months to 1 year 1 to 2 years continue to meet standards and follow protocol

NCAAA QUALITY PLANNING AND REVIEW CYCLE

Define/ReviewMission/Goals

Environment Scan

Five yearly Review

Review Performance

Action Plan

Set Objectives

Monitor Results

Implement Plan

Develop Plan

Yearly


Institutional and Program Accreditation

Accreditation is public recognition that quality assurance Standards are met in the management and delivery of learning outcomes achieved by students.

Standards must be equivalent to high quality international Standards.


Accreditation Process

Process involves a rigorous self evaluation in relation to

Institution Goals + the eleven NCAAA Standards followed by an external review.

An external Review Panel will verify the conclusions of the Institution’s self evaluation according to its quality of performance in relation to the Standards.

If an Institution is to be accredited then ALL eleven Standards must be met (including Mission and Goals).


Accreditation means….

TRUST It means that we can trust the university,

institution, college, program, or department

It means the institution or program is doing what it says it is doing.


Foundations of Accreditation

• Trustworthiness• Commitment• Quality Standards• Common Protocol• Transparent • Internal Self Evaluation• External Evaluation / Peer Reviews• Continuous Improvement

INSTITUTIONAL SELF STUDY

• Primary focus of External Review• Takes place before accreditation• Institutional Quality Unit and Committee• Consideration of performance• Report and conclusions


NCAAA: “the BIG 6”

Internal Evaluation Includes:

SurveysSWOT AnalysisSpecificationsSelf-Evaluation Scales

Self Study Report (SSR)Strategic Action Plan

Purposes of Self Study

• A powerful systems approach to improving

student performance and

system effectiveness providing results over time


By conducting a data-driven-statistical Self-Study of its own strengths and opportunities for change, the institution or program will…

Involve the campus community to evaluate its mission, educational programs, activities, ongoing processes of planning, resource allocation, and university renewal; SURVEYS

involvement = investment

Purposes of Self Study

• Identify Strengths and weaknesses

• Framework for Accreditation Processes

• Enhances Development and Effectiveness

Purposes of Self Study Report (SSR)


Provides framework for accreditation

‘should provide sufficient description to enable the review team to understand the key features of the institution’s approach to assuring quality and securing standards, but should focus on the effectiveness of that approach. Where the institution expresses confidence in its own effectiveness, the evidence upon which its view is based should be made clear’

‘The periodic institution self study provides the basic resource for the external review for re-accreditation .’

NCAAA: Handbook for Quality Assurance and Accreditation

DESCRIPTION – ANALYSIS – EVIDENCE – REFLECTION – ACTION



Enhance the shared understanding of its mission as an educational university or college; UNITY and team play

Ensure that the university’s or college programs and activities strive towards a common vision; all on the same spacecraft going to the same planet


Provide a framework for continuous improvement of the institution;

Initiates an attitude or culture of growth and improvement


Provide comprehensive, tangible, and coherent recommendations for the university’s future plans—recommendations that have been developed, evaluated, and committed to by the entire community; are comprehensive and measurable VS

personal agenda



SSR WAY FORWARD

‘Periodic self-study examines the institution in greater depth, checking on how effectively it is achieving its mission and objectives, and planning for any changes that are needed.’

‘A self-study should also consider how an institution has evolved over time in response to evaluations and changing circumstances and the probability that it will continue to be refined and improved in the future. Consequently developments over the period of review need to be considered as well as quality issues at the time of the self-study.’

NCAAA: Handbook for Quality Assurance and Accreditation


SSR Stakeholders

University leaders institutions

Program leaders colleges / departments

Professors classrooms

Students enrolled and alumni

Public employers and community

MOE, MHE, and government leaders

PROPERTIES OF A SELF-STUDY REPORT (SSR)

‘Many self-study documents (report) were found to be frank, honest and self-critical, though occasionally criticisms directed by institutions at their own arrangements for quality and academic standards were found to have overstated their deficiencies. Conversely, in a few cases, other self-study documents were found to have overstated the strengths of the institution's arrangements.’

QAA, 2007

Institution Eligibility for Applying for Accreditation and Conducting SSR

Institutional Eligibility Check List CriteriaCriteria

MetConfirmed

)NCAAA(1. Final license or approved government institution2. Activities consistent with license or approval3. Approved and consistent with license or approval 4. Strategic plan including plan for quality5. Availability of policies, regulations and terms of reference6. Published guides or handbooks for students7. Program specifications for all programs8. Course specifications 9. Regulations and descriptions of processes for program approval,

changes, and review10. Systems for monitoring quality and improving programs11. Central maintenance analysis and reporting of statistical data12. Student surveys13. Quality assurance system covering all standards14. Data on Key Performance Indicators15. Arrangements for comparative benchmarks16. Systems for maintenance of data on research (if applicable)17. Systems for maintenance of data on community service activities18. Students graduated19. Compliance with standards for accreditation.

Group Activity 120 minutes

(10 minutes for discussion and 10 minutes to report)

Has your institution met eligibility requirements?

Use the Checklist provided.


Day 1

S 3 S 2 S 1


evidencePlanning the

SSR



SSR STARTING POINT

1. First, read and analyze the NCAAA standards;

2. Second, make a commitment to meeting NCAAA Standards and protocol.

3. Third, assess the University or Program status in relationship to the NCAAA standards


SSR is WORK…WORK…and more WORK

1. TIME is needed for university staff to do all the work that is required. Deans must consider reduced teaching and/or research loads, or additional compensation

2. Strong and meaningful support is needed from each college dean and university administrator to create enthusiasm and positive attitudes.


PLANNING THE SSR1. What are we trying to do?

2. Why are we doing it?

3. How are we doing it?

4. Why is that the best way to do it?

5. How do we know if it works?

6. How do you change if need to?

7. How evaluate success of change?


PREPARATION OF INSTITUTION Self Study– CONSIDERATIONS?

• Led by senior member of faculty• Guided/approved by a Steering Committee• Working parties• Training• Buy-in of Senior Staff• Annual reports• Editorial control• Benchmarks• Timing


PROCESS FOR SS

• Resources• Communication• Independent analysis and comments• Concluding statement and Action

Plan


KEY is the FACULTY

NCAAA accreditation and the whole university or college level of success is directly tied to the faculty.

The faculty is the fulcrum for quality, a SSR, and for academic success.

Maximum faculty involvement Strong faculty = Strong university


SSR Leadership Team

NCAAA accreditation SSR is the work of the whole faculty team and not just 20% of the faculty.

Team LEADERS are needed…

LEADERSHIP is fundamental

SSR Accreditation Leadership (Deanship of Evaluation, Quality, and

Academic Accreditation)


1.Nominate Chair &

Coordinator

2. Appoint Steering

Committee

3.Plan Strategy for Self Study

5.Appoint Sub Committees

5. Special tasks that may need attention for the reviewProvide information about the self study to faculty, staff and students.

Coordinate activities for the institutional review and the program reviews to minimize duplication.

6.Review Performance on

Indicators

8.Gather Information Complete Self Eva

Scales

9. Sub Committees

Draft Reports

11. Report given to Rector and

Governing Board

12Report given to NCAAA

for External Review

10Self-Study Report

Prepared

Steps in Conducting an Institutional Self Study

SSR Accreditation Leadership Team

The Accreditation Leadership Team consists of key individuals that will drive the Self-Study process to a successful completion.

Deanship of Evaluation, Quality, and Academic Accreditation

. Its primary task is to plan, guide and coordinate the Self-Study Report (SSR). It’s responsibilities are:

1. Appointing necessary subcommittees, 2. Preparing the Self-Study Report with all supporting

documents and evidence, 3. Setting the schedule for the Review Panel site visit. 4. Participating as hosts and resources to the Review Panel

team and to fellow-up with faculty members and administrators for sessions.

5. Reviewing the report of the visitation Review Panel.6. Make recommendations for continued improvement.7. Follow-up on Review Panel Report recommendations

SSR Leadership Chair

1.Keep everyone on task2.Ensure the SSR subcommittees are

functioning and carrying out their tasks ON TIME

3. Encourage and motivate those working with the accreditation process

4. Work closely with the university’s administrator5. Attend the pre-visit with the NCAAA Visitation

Team and Liaison Officer6. Chair the SSR Leadership Team meetings7. Ensure that the work of the SSR TEAM stays on

the task timeline.


Qualifications of the SSR Leadership Team & Chair

Dedication to the Mission of the universityTrusted members of the universityFamiliarity with the university

No conflict of interest employment, either personally or of a family member, within the university Strategic planning, technological, and

organizational skillsStrong interpersonal skills

TEAM BUILDER


• Based on Institution size, type, teams• Based on Standards, Group of

Standards, or functions

Forming Steering Committee

Decide on Model of Self Study

Defined a detailed Plan with responsibilities and timeline


Steering Committee

Sub-committee 1

Standard 1 Standard 2 Standard 3

Sub-committee 2

Standard 4

Sub-committee 3

Standard 5 Standard 6

Sub-committee 4

Standard 7 Standard 8 Standard 9

Sub-committee 5

Standard 10Standard 11

Sub-committee 6Follow up -

coordination

Forming Steering Committee:Example

Finding the Story:Meaning and Interpretation

Self-STUDY Examination and Observation

How do we collect and arrange the data? DESIGN

How do we interpret the data? MEANING

What does the meaning imply? IMPLICATIONS How do we apply the data? ACTION?


Thinking about SSR

We need to understand that NCAAA accreditation and writing a SSR is not just meeting Standards and following a process protocol.

We need to understand that NCAAA accreditation and writing a SSR are a comprehensive quality assurance system.

System Elements

4545

Transportation

Technology MIS

Executive StaffPrograms

Accounting Finance

ClassifiedStaff

LocalBusiness

Community

Students

Parents

Teachers

Colleges & Universities

Facilities & Maintenance

Food Service

Systemic University Improvement

Alignment

At

ALL

Levels

University

College Program

DEPARTMENT

CLASSROOM

STUDENTS

IMP

RO

VE

ME

NT

S

All-Comprehensive Institutional

System for Systems

Horizontal and

Vertical


SSR in Three Steps

Step One – StudyStep Two – AnalyzeStep Three - Show

Evidence


3 Steps of Self-Study

Step Two: Analyze Data, Descriptions, Quality Indicators, Evidence and Documents…

• In this step, the Institution does a self-rating based on the Standard and KPI Quality Indicators and analyzes the information.

• The analysis identifies findings, strengths, and recommendations (weaknesses and needs).

• After the analysis, intervention actions will address the needs that were suggested.


Reason: What rational?

What is the thinking that is behind the analysis or assessment?

How are the statistics, graphs, or tables interpreted or given meaning?

How is the MEANING applied?What is the logic behind the

evaluation?Copyrights © for materials are reserved for NCAAA

3 Steps to Self-Study

Step Three: Show Evidence • In this step, the Institution or Program

collects documents and artifacts that provide verification that the Standards are met and the process was scientific.

• This information is either included in the Self-Study Report or available for the Review Panel to evaluate during the visit.

• Interviews and observations take place during the accreditation review visit to provide professional opinions.


Support Evidence

What evidence supports that a quality indicator is met and evaluated accurately?

What evidence documents that the answers to the questions are accurate?


• What steps have you already taken to conduct your institution Self Study and to prepare your SSR?

• What difficulties have you experienced?

• How have you resolved these difficulties?




Day 1S 3 S 2 S 1


evidence

Planning the SSRIntroduction and

Purpose of the SSR


NCAAA Standards

1) Mission and Objectives2) Governance and Administration3) Quality Assurance and Improvement4) Learning and Teaching5) Student Administration and Support

Services6) Learning Resources7) Facilities and Equipment


NCAAA Standards

8) Financial Planning and Management

9) Faculty, Staff and Employment Processes

10)Research

11)Institutional Relationships with the Community


Using Self Evaluation Scales (SES)

Initial Self Evaluation

Continuous Monitoring

Self Study

Guide External Review Panel

Mid Periodic Reviews


Using Scales

How well is this done?

(enter stars)

NA

N

Y

Is this true?Y/No/NA

-------

0

***

*******

*****

Improvement Required

Good Performance

High Quality Performance

11 Standards

Sub- Standards

Self Evaluation Scales (SES)


Using stars for evaluations Yes/No/Not applicable

IMPROVEMENTS REQUIREDNo star - practice, though relevant, is not followedOne star - practice followed occasionally but quality is poor or

not relevantTwo stars -practice usually followed but quality is less than

satisfactory.

GOOD PERFORMANCEThree stars - practice followed most of the time

- evidence of the effectiveness of activity is usually obtained

- evidence indicates satisfactory performance normally

achieved - plans for improvement in place and progress monitored



HIGH QUALITY PERFORMANCE

Four stars- practice consistently followed

- indicators of quality of performance are established

- mainly high quality performance - still some room

for improvement

- plans for improvement implemented and progress

monitored and reported

Five stars - practice is followed consistently

- high standard supported by evidence

- evidence indicated superior to comparable

institutions

- realistic strategies for improvement



Good Practices Relating to Standard 3: Quality Management

3.1 Institutional Commitment to Quality Improvement An institution must be committed to maintaining and improving quality through effective leadership and active involvement of teaching and other staff. 3.1.1 The Rector or Dean strongly supports involvement in quality assurance processes. 3.1.2 Adequate resources are provided for the leadership and management of quality assurance processes, and provision of assistance where it is needed. 3.1.3 All teaching and other staff participate in self-assessments and cooperate with reporting and improvement processes in their sphere of activity. 3.1.4 Creativity and innovation combined with clear guidelines and accountability processes are actively encouraged at all levels. 3.1.5 Mistakes and weaknesses are recognized and used as a basis for planning for improvement. 3.1.6 Improvements in quality are appropriately acknowledged. 3.1.7 Evaluation and planning for quality improvement are integrated into normal administrative processes.

Overall Assessment Comment_____________________________________________________________ _____________________________________________________________________ Priorities for improvement_______________________________________________ ____________________________________________________________________ Independent Opinion Comment_____________________________________________________________

_____________________________________________________________________


How well is this done?(enter stars)

***yes

****yes

***yes

0no

**yes

***yes

****yes

***

***

***yes

4.1 Student Learning Outcomes Intended student learning outcomes must be consistent with the National Qualifications Framework, and with generally accepted standards for the field of study concerned including requirements for any professions for which students are being prepared. 4.1.1 Intended learning outcomes are specified after consideration of relevant academic and professional advice. 4.1.2 Intended learning outcomes are consistent with the Qualifications Framework. (covering all of the domains of learning at the standards required) 4.1.3 Intended learning outcomes are consistent with requirements for professional practice in Saudi Arabia in the fields concerned. (These requirements should include local accreditation requirements and also take account of international accreditation requirements for that field of study, and any Saudi Arabian regulations or special regional needs.) 4.1.4 If an institution has identified special attributes to be developed in students graduating from the institution comprehensive strategies are established for these to be developed. (This means that the attributes to be developed in students are clearly defined, strategies for developing them planned and implemented across all programs, and mechanisms for assessing and reporting on the extent to which graduating students have developed them, are in place.) 4.1.5 Appropriate program evaluation mechanisms including graduating student surveys, employment outcome data, employer feedback and subsequent performance of graduates are used to provide evidence about the appropriateness of intended learning outcomes and the extent to which they are achieved. (see also sections 4.3 and 4.4 dealing with verification of standards and program evaluation processes) Overall Assessment Comment_______________________________________________________________ _______________________________________________________________________ Priorities for improvement_________________________________________________ ______________________________________________________________________ Independent opinion Comment_______________________________________________________________

***yes

***yes

****yes

***yes

***

***



4.3 Program Evaluation and Review Processes The quality of all courses and of the program as a whole must be monitored regularly through appropriate evaluation mechanisms and amended as required, with more extensive quality reviews conducted periodically. 4.3.1 Courses and programs are evaluated and reported on annually with information about the effectiveness of planned strategies and the extent to which intended learning outcomes are being achieved. 4.3.2 When changes are made as a result of evaluations details of those changes and the reasons for them should be retained in course and program portfolios. 4.3.3 Quality indicators that include learning outcome measures are identified and used for all courses and programs. 4.3.4 Records of student completion rates in all courses and the program as a whole are kept and used as quality indicators... 4.3.5 Quality indicators for programs are kept on regularly reviewed by senior administrators and quality committees. 4.3.6 A selection of quality indicators including progression and completion rates are compared across the institution and by reference to appropriate external benchmarks, and action taken when problems are identified. 4.3.7 If problems are found through program evaluations appropriate and timely action is taken to make improvements 4.3.8 In addition to annual evaluations a comprehensive reassessment of the program should be conducted at least once every five years. Procedures for conducting these reassessments should be consistent with policies and procedures established for the institution. Overall Assessment Comment_______________________________________________________________ Priorities for improvement_________________________________________________ Independent opinion Comment_______________________________________________________________

***yes

***yes

***yes

****yes

**yes

***yes

***yes

***yes

***

***



Evidence Wheel

Surveys

SWOTAnalysis

Program &Course

SpecificationsSelf

StudyReport

StrategicPlan

NCAAAAccreditation

Profile

Evidence


EVIDENCE

What EVIDENCE is needed to support that this Standard or quality indicator is met?

What REASON or RATIONAL is given to support that the quality indicator is met?

What DATA or STATISTICAL DOCUMENTATION is needed to support this quality indicator is met?

List evidence after each quality indicator on the lines provided

How are we going to get information???

What methods are used?1. Statistical Surveys WHO? For WHAT?

2. Historical Records WHAT? For WHO?

3. Structured-Unstructured Interviews WHO? WHAT to ASK?

4. Observations WHAT DO TO SEE? WHY?

5. Questionnaires WHO? WHAT to ASK?


Two Kinds of Data

Quantifiable Data (objective)What is this? How do we gather it? How do we use it for NCAAA accreditation?

Qualitative Data (subjective)What is this? How do we gather it? How do we use it for NCAAA accreditation?


Apply to…. Self Study Report

Profile: Data, statistics, description

Standards Self Study ReportCourse Specifications and Report

Program Specification and Report

Strategic Plan and Action Points

Evidence portfolio


Basic Questions for Interpretation are….

Ask: How does this information relate to our GOALS? …our VISION?

Ask: How does this information align to our core values and beliefs?

Ask: What strengths and what weaknesses do we see?

Ask: What problems still need to be resolved?

Ask: What additional information is needed?


Basic Questions for meaning and application are…

Ask: How do we improve?

Ask: What immediate action is needed?

Ask: What context or perspective is required for this information to be understood?

Ask: What are the assumptions or givens or presuppositions of this information?

Ask: What is the next step?

Ask: What bias or prejudices are present?


Key PERFORMANCE Indicators and Benchmarks

What is a Key Performance Indicator (KPI)?

KPIs are intended to provide specific quantifiable data that can be used to monitor quality and become selected for benchmarking purposes.


Examples of Benchmarks

1. 15 students for every 1 teaching staff

2. 75% of the teaching staff with verified doctoral qualifications

3. 25.5 book titles held in the library per student enrolled

4. 0.75 computer terminals per student enrolled


SOURCES OF EVIDENCE for SSR?


STUDENT EVIDENCE

• Achievement– Completion/wastage rates/retention– Entry qualifications– Academic distance travelled (value added)

• Experience– Contact hours– Staff/Student Ratios– Complaints/Appeals– Selected Student Survey Indicators


STUDENT EVALUATION SURVEYS

• Anonymous• Common question sets• Open ended questions• Trend analysis• Frequency and Length• Course Evaluation Survey (After completion of each course)

• Student Experience Surveys (After completion of half the program)

• Program Evaluation Survey (After/ at the last semester of program)


Course Evaluation Survey (CES)

Course Title _______________________________Program________________________________

Semester__________________________________Year___________________________________

Feedback from students is very important in trying to improve the quality of courses. This is a confidential survey. Do not write your name or identify yourself in any way. Your responses will be combined with the responses of others in a process that does not allow any individual to be identified and the overall opinions will be used to plan for course improvements. Please respond to the following questions by completely filling a response for each of your answers.

Do this Do not mark in any of these ways √ × • Use a pencil or blue/black pen only Do not use red, green or yellow Make heavy marks that fill in your response Do not use highlighters

Questions about the start of the course:

1. The course outline (including the knowledge and skills the course was designed to develop) was made clear to me.

2. The things I had to do to succeed in the course, including assessment tasks and criteria for assessment, were made clear to me.

3. Sources of help for me during the course including faculty office hours and reference material, were made clear to me.

Strongly agree means the statement is true all or almost all of the time and/or very well done.

Agree means the statement is true most of the time and/or fairly well done. True sometimes means something is done about half the time. Disagree means something is done poorly or not often done. Strongly disagree means something is done very badly or never or very rarely done.


Questions about what happened during the course:

4. The conduct of the course and the things I was asked to do were consistent with the course outline.

5. My instructor(s) were fully committed to the delivery of the course. (Eg. classes started on time, instructor always present, material well prepared, etc)

6. My instructor(s) had thorough knowledge of the content of the course.

7. My instructor(s) were available during office hours to help me.

8. .My instructor(s) were enthusiastic about what they were teaching

9 My instructor(s) cared about my progress and were helpful to me.

10. Course materials were of up to date and useful. (texts, handouts, references etc.)

11. The resources I needed in this course (textbooks, library, computers etc.) were available when I needed them.


12. In this course effective use was made of technology to support my learning. 13. .In this course I was encouraged to ask questions and develop my own ideas

14. In this course I was inspired to do my best work.

15. The things I had to do in this course (class activities, assignments, laboratories etc) were helpful for developing the knowledge and skills the course was intended to teach.

16. The amount of work I had to do in this course was reasonable for the credit hours allocated.

17. Marks for assignments and tests in this course were given to me within reasonable time.

18. Grading of my tests and assignments in this course was fair and reasonable.

19. The links between this course and other courses in my total program were made clear to me.

Evaluation of the Course 20. What I learned in this course is important and will be useful to me.

21. This course helped me to improve my ability to think and solve problems rather than just memorize information.

22. This course helped me to develop my skills in working as a member of a team.

23. This course improved my ability to communicate effectively. Overall Evaluation


24. Overall, I was satisfied with the quality of this course.

25. What did you like most about this course?

26. What did you dislike most about the course?

27. What suggestions do you have to improve the course?


KPI Quality Indicators

What quality indicators or KPIs support or verify that a Standard is met?

What questions should be answered to indicate that the Standard is met? (What evidence and for what reason?)


Example: Mean Rating by Final Year Students on Q 21 of Program Evaluation Survey

(Q21: I have developed the knowledge and skills required for my chosen career)

College 2007-8 2008-9 2009-10Male Female Male Female Male Female

Eco & Com 2.98 3.43 3.01 3.55 3.12 3.61Education 3.22 3.16 3.30 3.08 3.63 3.22Science 3.60 3.65 3.50 3.49 3.58 3.61Islamic Stud 4.12 3.89 4.00 3.80 3.96 4.02Medicine 3.31 3.58 3.76 3.92 3.85 3.89Nursing 3.22 3.88 3.33 3.91 3.26 3.98University 3.55 3.72 3.77 3.80 3.76 3.84

Questions1.Which (if any) of these results would you regard as satisfactory?2.What target(s) could you set for improvement?3.What strategies could you suggest to improve the results of some

colleges?


A- Examine Standard 4 and create KPI quality indicators for it in the connecting circles.

B- Complete the Self Evaluation Scales for a Sub-Standard under this Standard for your institution

C- Think about the following:– What evidence would you use to complete the

scales?– From where would you obtain this evidence?– Based on your experience, what difficulties do you

expect in collecting and using evidence and KPIs?




Standard 4Learning and

Teaching

4.2Student Learning

Outcomes

4.6Educational Assistance

for Students

4.7Quality of Teaching

4.5Student Assessment

4.3Program Development

Processes

Representative Substandards

4.2 Student Learning Outcomes

Intended student learning outcomes must be consistent with the National Qualifications Framework, and with generally accepted standards for the field of study concerned, including requirements for any professions for which students are being prepared. Programs must be planned in a way that ensures that all courses contribute to program learning outcomes in a coordinated way.


4.2 Student Learning Outcomes

KPIs

1. Ratio of Students to Teaching Staff

2. Percentage of students entering programs who successfully complete the first year3. Proportion of graduates from undergraduate programs who within six months of graduation are:

• Employed• Enrolled in Further Study• Not Seeking Employment or Further

Study


How do we use KPIs to determine how well we perform?

Benchmark 30:1

Benchmark 85%

1. Ratio of Students to Teaching Staff

2. Percentage of Students entering Programs who Successfully Complete the First Year

University 27:1

University: 77%

What does th

is

mean?

What Do w

e do about it?


% Student Completion1st Year

2011

2012

2013

2014

0

10

20

30

40

50

60

70

80

90

100 UniversityBenchmark

Academic Year


End of Day 1


PLANNING AND IMPLEMENTATION OF SELF-STUDY

for Institutional Accreditation

Day 2


Day 2S 3 S 2 S 1


Sharing Experiences


Conclusions

Structure and Content of the

SSR


MATTERS FOR INCLUSION IN AN INSTITUTIONAL SSR

• Institutional Profile• Context – internal and external• Mission – Goals – objectives• Evaluation in relation to Quality

Standards (KPIs+ Benchmarks)• Conclusions and Recommendations


• General Information• Self study process• Mission and goals of the

programme• Key performance indicators [KPIs]

MATTERS FOR INCLUSION IN SSR


CONTENT OF SSR

• Authorative – evidence based• Evaluative – analyse and reflect• Fit for purpose – required coverage• Constructive – enhancement and good

practice• Action plan – by whom and when


INTERPRETING SELF EVALUATION SCALES

• Broad evaluation possible?• Particular importance attached to what student

learnt• Have Learning Objectives been met?• Standards of student achievement• Benchmark against standard• Can evidence be verified?


A. General Information

Name of InstitutionDate of Report

B. Institutional ProfileAn institutional profile should be prepared including the following material: A brief summary of the institution’s history, scale and range of activities; A description of the management and organizational structure using an organizational

chart, list of colleges and departments, and the names and contact details of key individuals;

A list of campus locations indicating programs offered and student numbers; Teaching and other staff and student numbers in total and by college, department, and

program; Summary information about the institution’s accreditation status including the

outcomes of any previous institutional reviews, and any conditions that were established;

A description of the institution’s quality assurance arrangements, priorities for development, and any special issues affecting its operations;

A summary of the institution's strategic plan. (A copy of the actual strategic plan should be available for reference if required.)

A list of matters that are of particular interest to the institution and on which the institution is seeking comment and advice in the review.

Handbook for quality assurance and accreditation in Saudi Arabia

SSR Template

What is the Profile?

A picture of the University using data…

Using data to improve what you are doing!

• University Descriptions

• List Accomplishments

• Enrollment Data – statistical trends over several years, with predictions



A picture of the Institution using dataUsing data to improve what you are

doing!

Student Academic Achievement and

Performance

OutcomesCopyrights © for materials are reserved for NCAAA


A profile is assessment data….

Student Achievement Test Scores or licenses earned…

Failures/Drop-out rates Acceptance rates, profile of students Follow-up on alumni or graduates

Comparative statistics

Satisfaction of University or College Programs Survey Statistics and Analysis

C. Self-Study Process

Provide a brief description of procedures followed and administrative arrangements for the self study. Include an organization chart. Membership and terms of reference for committees and /or working parties should be attached.

D. Context of the Self Study

1. Environmental Context.Summary of significant elements of the external environment in which the institution is operating and changes that have occurred recently or are expected to occur (eg. economic or social developments, population changes, government policies, developments at other institutions with implications for this institution’s programs).

2. Institutional Context.Brief summary of recent developments at the institution with implications for the review.

Handbook for quality assurance and accreditation in Saudi Arabia

SSR Template

E. Mission, and Goals and Strategic Objectives for Quality Improvement

1. Mission of the Institution

2. Summary of Strategic Plan for Quality Improvement ( The institution's major goals and strategic objectives for quality improvement should

be listed, indicating for each objective, performance indicators and benchmarks or standards of achievement the institution wishes to achieve.)

F Special Emphasis in the Self-Study (if any)

Indicate any areas of particular interest to the institution in the review. (These may relate to responses to changes in the external or institutional environment, to planning priorities that may have been determined as a result of quality assessments or other strategic priorities, to government policies, etc.)

SSR Template

G. Progress Towards Major Quality Objectives (Refer to Item D2 above)

Assessment of institutional performance in relation to plans or any major quality improvement initiatives in the period under review. These may have been undertaken in response to a previous self study, recommendations or requirements following an external review, or for other reasons.Brief reports should be provided on each major initiative citing the objective(s), specific data indicating the results achieved, and a comment on reasons for success or failure to achieve the desired results.

Objective 1 Result Achieved (Performance in relation to indicators and benchmarks)Comment Objective 2 Result Achieved (Performance in relation to indicators and benchmarks)Comment (continue for other strategic quality improvement objectives)

SSR Template


How you would consider assessing your institutional performance in relation to plans or major quality improvement initiatives in the period under review.

What indicators would you use? And Why?




H. Evaluation in Relation to Quality Standards

Reports should be given on performance in relation to each of the standards set out in the Standards for Quality Assurance and Accreditation of Higher Education Institutions. The reports should refer to areas of strength and weakness as indicated by the rating scales in the accompanying document—Self Evaluation Scales for Higher Education Institutions.

•Explanatory note about development and use of the mission.

•Description of process for investigation and preparation of the report on this standard.

•Report on subsections of the standard

•Overall Evaluation of Performance on standard. Refer to evidence obtained and provide a report based on that evidence and including a summary of particular strengths, areas requiring improvement, and priorities for action

1. Standard …….. (Overall Rating _______Stars)

SSR Template

H. Evaluation in Relation to Quality Standards

EXAMPLES

Writing Strengths-Areas for Improvement- Priorities for Improvement

SSR Template


Enrollment

0

10

20

30

40

50

60

70

80

2002 2003 2004 2005

Sophomore

Junior

Senior

Strength:• Senior enrollment

Weakness:• Enrollment

fluctuation• Revenue instability• Class size change

Profile Sample

Findings:•3rd enrollment decreases every year

•3rd decline causes future enrollment problems

•Senior enrollment has steady increase

• Junior enrollment is up and downCopyrights © for materials are reserved for NCAAA

Examples of Reports on Progression RatesProgression Rate from First to Second Year –

(Proportion of all students enrolling in first year in each college who continued to the second year of their program the following year)

Colleges 2007-8 2008-9 2009-10Male Female Male Female Male Female

Eco & Com 48.38 55.33 45.8 56.98 50.1 52.3Education 66.3 78.34 62.87 76.44 63.98 77.32Engineering 41.33 44.36 42.11Law 55.33 62.29 58.35 49.35 46.1 51.23Science 57.83 59.22 53.52 56.91 55.3 57.26Medicine 58.91 61.33 57.34 63.89 61.2 66.87Nursing 58.22 55.13 56.81 59.16 54.35 58.2University 48.38 55.33 45.8 56.98 50.1 52.3

Questions

Would you consider the proportion of students who complete first year courses but do not proceed to be a problem? How could you investigate the reasons for this drop out rate? What improvements are needed?


Mean Rating by Students on Satisfaction with the Quality of Their Courses(All Students, 2007/8 t0 2009/10)

Colleges 2007-8 2008-9 2009-10Sem 1 Sem 2 Sem 1 Sem 2 Sem 1 Sem 2

Eco & Com 3.03 3.29 2.98 3.25 3.13 3.33Education 3.32 3.11 3.20 3.29 3.09 3.22Engineering 3.70 3.75 3.68 3.69 3.68 3.65Law 4.12 3.89 4.00 3.80 3.96 4.02Science 3.01 2.98 3.16 3.22 3.45 3.49Nursing 3.62 3.78 3.77 4.01 3.99 4.02University 3.55 3.48 3.42 3.56 3.61 3.68

Questions1.Which college do you believe has the most satisfying results?2.What action need to be taken to improve the ratings by students? What

target would you set for improvement?




Complete your Self Evaluation Scales by: - Identifying KPIs and evidence used, - Areas of Strengths - Priorities for improvement


Day 2

S 3 S 2 S 1


Sharing Experiences


Conclusions



Writing Recommendations

• Write Priorities• Write Recommendations• Write Interventions

Recommendations Are…

A recommendation is written sentence with a challenging verb that directs tangible, measurable, or quantifiable action that is directed toward improvement.

Verbs with clear reference to measurable action

SECTION I SSRINDEPENDENT EVALUATION

• Comment on relative standards• Critical friend• Summarise matters raised• Opportunity to respond

• Outside Institution has experience • Verify Evaluation Accuracy and results

interpretations based on evidence

I Independent Evaluations

1. Describe the process used to obtain independent comment on the self study.

Processes may include a review of documentation by experienced and independent persons familiar with similar institutions and who could comment on relative standards, consultancy advice or a report by a review panel, or even the results of an accreditation review by an independent agency. An independent evaluation may be conducted in relation to the total self-study, or involve a number of separate comments by different people on different issues.

2. Summary of matters raised by independent evaluator(s)

3. Comment on matters raised by independent evaluator(s) (Agree, disagree, further consideration required, action proposed, etc.)

Developmental ReviewMock Review


1. List and briefly describe institutional activities that are particularly successful or that demonstrate high quality.

2. List and briefly describe institutional activities that are less than satisfactory and that need to be improved.

J Conclusions

K Action RecommendationsThese should be based on the matters identified earlier in the report for further improvements or to overcome problems or weaknesses identified indicate specific actions proposed to deal with the most important priorities for action identified in those sections. Matters of greatest urgency or highest priority should be identified. For each action proposed recommendations should be made on who should be responsible for the action, timelines specified, and any necessary resources required.Action Recommendation 1 …..

Person (s) responsibleTimelines (For total initiative and for major stages of development)Resources Required

CONCLUSIONS

Base SSR preparation on Annual Monitoring Reports

Use for Quality Assurance and Quality Improvement

– AUTHORATIVE – evidence based– EVALUATIVE – analyse and reflect– FIT FOR PURPOSE –coverage– CONSTRUCTIVE - good practice and improvement– ACTION PLAN – responsibilities and time lines


R A C I

R - Responsible – One who owns the project

A - Accountable -To whom R is responsible Must sign off on the work before it becomes effective, veto (yes/no) power

To be Consulted - Has information/or capacity necessary to complete the work

To be Informed - Must be notified of the result but not consulted

Activity Faculty Member Department Chair Vice Rector Rector

Learning ResourcesR A C I

Facilities and EquipmentR ? ? ?

Learning and Teaching C R A I

Student Administration and Support Services

? R ? ?

Faculty and Staff Employment Processes

? ? R A

R A C I -Example


ACTION PLANS GENERAL

• Rolling or fixed date• Tabular• Assign responsibility• Base on reality – recorded issue leading to

achievable solutionS – Specific

M – Measurable

A – Achievable

R – Responsible – Result oriented

T – Time lines


ACTION PLAN SPECIFIC

• Sign off Action Plan • Changes • Staff development needs• Seize initiative

LINK ALL ACTION POINTS TO

ISSUES RAISED IN ANNUAL REVIEWs


ACTION PLAN EXAMPLE

Action Date Responsibility Resources

1.

2.

3.

Evaluate if completed

Link to summaries in SES


The Institutional Self Study Report should be on A4 paper, unbound, printed on one side, page numbered, and with a table of contents for easy reference. Attachments Membership and terms of reference for sub-committees and working parties

Reference list of key reports and other documents cited in the report

Glossary of acronyms and abbreviations used in the report

Copy of report(s) by independent evaluator(s)

Documentation and Attachments


Self–evaluation Scales for Higher Education Institutions. The completed scales should include star ratings, independent comments and indications of priorities for improvement as requested in the document, and should be accompanied by a description of the processes used in investigating and making evaluations.

A copy of the institution's strategic plan.

A copy of the institution's strategic plan for quality improvement (which may be included within the broader institutional strategic plan).

Current student catalogue, prospectus, bulletin or handbook including descriptions of the curriculum, admissions requirements, degree completion requirements, and related information.

Documentation and Attachments with SSR


• • Faculty handbook or similar document with information about staffing policies,

professional development policies and procedures and related information.

• Administrative and financial policies manual or similar document including the institution’s bylaws and regulations, roles and responsibilities of administrative and academic officers and major committees, and an explanation of the institutions governance and administrative structure.

• Quality assurance manual or description of procedures including information about the institutions system of assessing programs and services, the role of the institution’s quality center and systems for gathering and analyzing data on quality of performance and planning for improvement.

• Current data on faculty and other teaching staff including tables with numbers by academic rank, by highest qualification, teaching staff/student ratios for each department and college, and for the institution as a whole.

• For a university (optional for a college) information should be provided on research output for each department, college and for the institution as a whole. Current teaching staff CVs should be on file and available for the review panel if required.

Documentation to be available in the review visit


Use the Ratings of the Scales and available documents and evidence to write an SSR report on Standard 4

1-Explanatory note about development and use of the mission.2- Description of process for investigation and preparation of the report on this standard.3-Report on subsections of the standard 4-Overall Evaluation of Performance on standard including: - summary of particular strengths, areas requiring improvement, - and priorities for action

Standard 4 Teaching and Learning (Overall Rating _______Stars)


(15 minutes for discussion and writing and 15 minutes to report)


Day 2S 3 S 2 S 1


Sharing Experiences


Conclusions



Continue Group Activity 6


Using your previous report, draw up an Action Plan which will lead to improvement in your institution regarding standard 4.



Actions Date Responsibility Resources

1.

2.

3.


CommonChallenges and Problems

Success Factorsin Conducting Institutional SSR


End

Thank you
