Program Evaluation Information and Procedures · program evaluation team member with the name, address, and phone number of a representative of the host programwho serves as the local

October 2019

International Fire Service Accreditation Congress

Degree Assembly

Program Evaluation Information and

Procedures

IFSAC Degree Assembly Program Evaluation Information and Procedures October 2019

TABLE OF CONTENTS

INTRODUCTION ..................................................................................................................... 1

THE PROGRAM EVALUATION TEAM ....................................................................................... 1

Selection ................................................................................................................................... 1 Empowerment: A By-Product of Being Selected ...................................................................... 2 Expense Procedures ................................................................................................................. 2 IFSAC Program Evaluation Team and Program Entity Code of Ethics ..................................... 3 Characteristics of Successful Program Evaluation Team Members ........................................ 4 Conflicts of Interest .................................................................................................................. 4 Issues of Consulting ................................................................................................................. 4 Public Disclosure and Confidentiality ....................................................................................... 5 Evaluation Team Leader Responsibilities ................................................................................ 5

PREPARING FOR THE ACCREDITATION EVALUATION .............................................................. 6

THE ACCREDITATION SITE VISIT ............................................................................................. 7

Arrival ...................................................................................................................................... 8 Site Team Work Area ............................................................................................................... 8 Preliminary Meeting with Program Staff ................................................................................ 8 Visit and Review of Support Services ....................................................................................... 8 Staff Interviews ........................................................................................................................ 8 Conducting Interviews ............................................................................................................. 9 The Exit Conference ............................................................................................................... 10 Deficiencies, Recommendations, and Requirements ............................................................. 11 Review of Report Forms ......................................................................................................... 12

AFTER THE SITE VISIT ........................................................................................................... 13

APPENDICES ........................................................................................................................ 14

APPENDIX A - PROCEDURES FOR SCHEDULING A PROGRAM EVALUATION .............................. 15 Institution Program Responsibilities ...................................................................................... 16 IFSAC Responsibilities ............................................................................................................ 16

APPENDIX B - MODEL PROGRAM EVALUATION AGENDA .......................................................... 17 IFSAC Onsite Team Schedule .................................................................................................. 18

APPENDIX C - MODEL PROGRAM EVALUATION REPORT ........................................................... 19 APPENDIX D - FINAL ACTION REPORT ........................................................................................ 23 APPENDIX E - DEGREE PROGRAM EVALUATOR SELECTION PROCEDURE .................................. 25

PURPOSE ................................................................................................................................ 26 QUALIFICATIONS .................................................................................................................... 26 SELECTION CRITERIA .............................................................................................................. 26 SELECTION PROCEDURE ......................................................................................................... 27

APPENDIX F - TIMELINE .............................................................................................................. 29

ENDNOTES .......................................................................................................................... 32

IFSAC Degree Assembly Program Evaluation Information and Procedures 1 October 2019

INTRODUCTION The purpose of a program evaluation for a program seeking initial accreditation or reaccreditation should be to gather objective data and well-grounded impressions regarding the program. The members of the program evaluation team should capitalize upon all feasible opportunities to observe characteristic operations of the program seeking accreditation/reaccreditation. IFSAC Administration (Administration), the Degree Assembly Board of Governors (the Board), and the Degree Assembly (the Assembly) will rely heavily upon presentations in context. The report developed by the program evaluation team should provide a clear and coherent account of the facts and impressions relating to the program actual operations.

THE PROGRAM EVALUATION TEAM Selection

The program evaluation team for initial accreditation consists of a minimum of five personnel: one team leader, two members, and two readers. The program evaluation team for reaccreditation consists of a minimum of three personnel: one team leader, one member, and one reader. Selections, where possible, are made to keep travel costs to a minimum, but are also based upon availability of individuals who have been approved and trained to participate in site visits. Evaluation team members should have no direct relationship, past or present, with the program that might be construed as a conflict of interest (see Degree Program Evaluator Selection Procedure, Appendix E). The evaluation team is to be selected 60 days prior to the site visit. Upon selection of the members of an onsite evaluation team, Administration communicates this information to the program hosting the visit and provides each program evaluation team member with the name, address, and phone number of a representative of the host program who serves as the local contact for coordination of travel. Administration also provides the names, addresses, and telephone numbers of the program evaluation team members to the program evaluation team leader and the host program. This should be done at least 60 days prior to the visit. In the event of unforeseen circumstances (sudden illness or injury of a site team member or his/her family, travel delays, etc.), with the Board’s approval, the number of approved program evaluation team members for an onsite evaluation may be reduced if another member is not available to attend the program evaluation. Should this occur, and the result is only one site visitor on site, a second evaluator should be available for consultation by electronic means. (Examples include conference call, Skype, GoToMeeting, or other electronic means).


Empowerment: A By-Product of Being Selected2

Accrediting bodies, as well as program evaluation team members, must realize that program evaluation team members benefit professionally from serving on teams. Some educators currently use "empowerment" as shorthand for this benefit. Onsite evaluation visits are rare learning opportunities. The learning that takes place often enables an onsite evaluation team member to become a more effective participant in his/her program. By granting these opportunities to individuals, accrediting bodies provide them with unique opportunities to enhance their skills and knowledge. IFSAC, by the very act of naming someone as a program evaluation team member, contributes to the professional stature of that person. Sometimes that added stature is evident only to those who are being visited; sometimes colleagues respond differently because of this apparent vote of approval. Empowerment, then, is not necessarily limited to what can be learned and applied; it can also involve enhanced respect. Program evaluation team members selected and trained by IFSAC should utilize this opportunity to the fullest extent. While avoiding the perception of "stealing" information, the learning experience itself should provide the program evaluation team members with a professional upgrade. Expense Procedures

Costs for onsite evaluations are the responsibility of the host program seeking accreditation. The purpose of the reimbursement process is to ensure that onsite program evaluation team members are expeditiously reimbursed for all reasonable expenses. It is the host program’s responsibility to provide the proper forms and make the reimbursement process as simple as possible. Onsite evaluation team members are entitled to receive payment for all direct travel-related expenses including expenses that incur due to circumstances beyond individual control (flight delays, hotel, and per diem cost due to weather delays). Evaluation team members do not receive compensation for their time. While not all institutions or states permit this, it is highly recommended that travel insurance be purchased when making airline reservations. When such purchase is permitted, this travel insurance shall be part of the site visitor’s reimbursable expenses. Host programs must reimburse onsite evaluation team members directly within 45 days following submission of appropriate expense vouchers. Rates and methods of reimbursement (mileage, lodging, and per diem) are based upon local regulations to which the host program must adhere. Host programs may also provide prepaid airline tickets to site team members and/or arrange for direct billing of lodging and/or meals. Lodging facilities must be equipped with full automatic sprinkler protection. The extent to which these arrangements are made is the prerogative of the host program.


IFSAC Program Evaluation Team and Program Entity Code of Ethics

The Code of Ethics applies equally to program entity representatives, all onsite evaluation team members, readers, observers, auditors, visitors, and anyone else accompanying the site team.

1. Members shall demonstrate the highest standards of personal integrity, truthfulness, honesty, and

fortitude in all dealings with program members, before, during, and after a program evaluation, to enhance fire science education internationally.

2. Members shall serve in a manner as to not receive undue personal gain from the performance of their official duties as IFSAC representatives.

3. Members shall avoid any interest or activity which is in conflict with the conduct of official duties as IFSAC representatives, such as serving as a consultant prior to or after a program evaluation, or accepting a position of employment from the program.

4. Members shall conduct their assessment of the program in a manner that ensures that the program is operating within their system in a fair and equitable manner.

5. Members shall treat program and fire service representatives with respect, concern, courtesy, and responsiveness, recognizing that the improved service of a program’s education system is more important than infliction of punishment for the sake of noncompliance.

6. Members shall seek professional excellence and development through continued training as a program evaluation team participant.

7. Members shall approach their duties as IFSAC representatives with a positive attitude and constructively support open channels of communication, creativity, dedication, and compassion when making decisions regarding program accreditation.

8. Members shall respect and protect the privileged information to which access is gained in the course of performing official duties as IFSAC representatives.

9. Members shall exercise whatever discretionary authority allowed by IFSAC bylaws to promote an accreditation process for degree programs that meet or exceed the various levels of competency identified in standards adopted by the Assembly.

10. Members shall respect, support, study, and when necessary work to improve IFSAC bylaws and criteria for accreditation for improved quality and efficiency in the program and better service for the constituency it serves.1

11. Programs will treat evaluation team members with respect, concern, courtesy, and responsiveness, recognizing that the improved service of a program’s education system is more important than infliction of undue pressure and argument to achieve accreditation.

12. Program representatives should seek professional excellence and development through continued training in the areas of preparing for and receiving an evaluation.

13. Programs shall approach the evaluation and evaluation team with a positive attitude and constructively support open channels of communication, creativity, dedication, and compassion when undergoing an evaluation.

14. Programs shall respect, support, study, and when necessary work to improve IFSAC bylaws and criteria for accreditation for improved quality and efficiency in the entity and better service the constituency it serves.

15. If the program and evaluation team and/or other IFSAC representatives are faced with unresolved issues where there is an obvious difference of opinion, it is the program’s responsibility to voice any concerns and/or appeal of such procedures in a civil, professional manner that is in accordance with IFSAC policy.

1 ASPA format for Code of Ethics Borrowed.


Characteristics of Successful Program Evaluation Team Members1 Background – Program evaluation team members have sufficient general education and special training specific to a professional discipline to form a solid foundation for program evaluation. Site team members may be either generalists or content specialists who are themselves practitioners or educators within the field of fire service training, education, and certification.

Program Evaluation Training – The International Fire Service Accreditation Congress (IFSAC) uses only program evaluation team members who have successfully completed an IFSAC sponsored program evaluation team member training workshop.

Attitude – Effective program evaluation team members demonstrate maturity, objectivity, diplomacy, and dedication. They project an image of professionalism both in behavior and appearance. Program evaluation team members appreciate the confidential nature of the task and understand the need for self-initiative, for a cooperative attitude, for an analytic approach to the task, and for necessary degrees of flexibility.

Knowledge – Effective program evaluation team members have a thorough understanding of fire education and of the accreditation process. They have sufficient general and special background to be able to exercise appropriate judgment.

Skills – Program evaluation team members are skilled in interviewing, interpersonal communications, self-expression, note-taking, and maintaining objectivity. They are skillful in dealing with attitudinal problems that may be presented by those being interviewed. Through experience and education, program evaluation team members have developed capacities for deductive reasoning and logical analysis. They are skilled in writing and accurate in recall.

Physical Factors – The physical condition of the program evaluation team members permits them to do whatever is necessary to conduct the onsite program evaluation in the particular locale and within the specified period. Conflicts of Interest3

All evaluation team members must be "impartial, objective and without conflict of interest." Some basic definitions appear to have wide acceptance: (1) team members will not be current or past employees of the institution/program being evaluated; (2) team members will not have graduated or enrolled in the program from the institution/program being evaluated; and (3) mentors who have assisted an program in preparing for their initial accreditation will not serve as program evaluation team members for the initial accreditation visit. IFSAC will not place on a team, individuals who have openly predetermined their decisions before the program evaluation. Unknown conflicts of interest can be discovered if an institution/program has the opportunity to respond to the recommended program evaluation team. Often this is the only way an accrediting body can learn that a prospective evaluation team member was a failed candidate for a position there, or had publicly said negative things about the institution or its programs. Each team member should also be reminded that even the appearance of conflicts of interest could weaken the credibility of the evaluation. Therefore, each program evaluation team member must take responsibility to inform the program of possible conflicts of interest. Issues of Consulting4


Program evaluation team members walk a thin line. In some accrediting bodies, they are encouraged to consult as well as evaluate. IFSAC encourages a frank discussion, and the opportunity for program evaluation team members to offer suggestions to the program. One appearance of apparent conflict of interest, however, is when a program, soon after the visit, hires one of the program evaluation team members as a consultant. IFSAC requires a grace period (at least one year) before program evaluation team members can serve as consultants for an institution/program they just evaluated. Any program evaluation team member who appears to be using program evaluations as a way to establish a consulting clientele will be removed from the pool of eligible program evaluators. If team members intend to do consulting business with the involved program in the future they should voluntarily remove themselves from the team. Public Disclosure and Confidentiality

It is critical for a program evaluation team to maintain the confidence of the institution and people within it. Team members often learn things that are not meant to be shared. The program evaluation team members must protect that confidence on site and away from it; the latter is as critical as the former. Program evaluation team members cannot interject themselves into the problems of an entity or program. They can observe, assess, and describe these problems, but they are not there to solve them during the site visit. If it is not published in the program evaluation report, keep it to yourself. It is imperative that program evaluation team members be viewed as impartial, professional, and sincerely interested in the success and quality of both IFSAC and the program. Onsite evaluation team members must be very sensitive to their language, both when soliciting information and when giving opinions, especially when discussing evaluative issues and observations regarding the program’s compliance with IFSAC accreditation criteria. Words with negative connotations should be avoided, as well as reprimands and lecturing, when ascertaining how faculty, students, and others perceive the program, its policies, and processes. Onsite evaluation team members should strive, through both verbal and non-verbal communication, to make the persons with whom they are talking feel comfortable about discussing the relative strengths and areas of concern, as well as what they contribute to or receive from the program. If notes are taken during the interviews or discussions, they should be recorded unobtrusively to avoid interfering with developing and maintaining good rapport.5 Evaluation Team Leader Responsibilities

The evaluation team leader has responsibilities beyond those of the other team members. These responsibilities primarily lie in the coordination and reporting of the evaluation. Communication is the key when working with both the program and the other team members. Scheduling the Site Visit


Although Administration will schedule the onsite visit, the team leader will be contacted for availability.. Schedule adjustments should be coordinated with other team members Working Out the Schedule Once the dates of the visit are established, it will be necessary for the team leader to contact the program and plan an agenda or schedule. A model agenda is contained in Appendix B. Contacting and Communicating with Team Members Constant communication between the team leader and evaluation team members is needed throughout the process on items such as schedules, reports, and additional information. Program evaluations are an ongoing process, as should be communication. Contacting and Communicating with the Program Constant communication between the team leader and the program is needed throughout the process on items such as schedules, reports, and additional information. Program evaluations are an ongoing process, as should be communication. The team leader becomes the point of contact for the program once the evaluation process has begun, until DABOG action. The team leader must communicate completely and effectively with the program on all items. Communication with Administration and DABOG The team leader is the voice of the team when presenting reports and information to the DABOG. Individual team members should communicate their findings to the team leader, who will then forward them to Administration. Approval of the Report Once the report has been approved by the team members, the team leader will make a report and recommendation to the Board. The report must conform to the requirements outlined in the IFSAC Bylaws and adopted procedures.

PREPARING FOR THE ACCREDITATION EVALUATION Administration will schedule all program evaluations. Onsite visits are scheduled upon the program’s completion of all requirements listed in the Assembly bylaws and Degree Assembly Procedure for Scheduling a Program Evaluation (Appendix A). At the time a program evaluation is requested, the host program must provide the name, address, and telephone number of a local contact person who can assist the onsite evaluation team members with travel arrangements. It is the responsibility of the host program to contact the individual onsite evaluation team members in order to coordinate


travel plans, lodging, and other logistical aspects of the visit. This should be done well in advance of the scheduled site visit (two to six weeks). Each evaluation team member should carefully review the Criteria for the Accreditation of Degree Programs of the IFSAC Degree Assembly prior to participating in an evaluation. Evaluation team members should also study applications and support materials which have been submitted by the program seeking accreditation to determine the apparent degree of compliance with the Criteria for the Accreditation of Degree Programs of the IFSAC Degree Assembly. A self-study is required to be included in your information. Review it carefully, since their item analysis is very similar to the items you will be studying during the visit. In addition, evaluation team members should review the Degree Assembly Procedure for Scheduling a Program Evaluation (Appendix A) to be sure that they have all the necessary materials for the visit. A notation of questions and concerns should also be made to speed up the interview and information gathering tasks during the visit. The program evaluation team leader should identify key program personnel and confirm with the host program their availability for interviews during the visit. The program seeking accreditation will distribute the self-study and support materials to the members of the onsite evaluation team at least forty-five (45) days prior to the scheduled visit. The program’s application for accreditation shall include all materials essential for review. If it is determined that additional materials are needed prior to the visit, IFSAC Administration shall make this request to the program, in writing, in advance of the site visit. IFSAC Administration reviews the application packet for completeness before handing it over to the Evaluation Team Leader. The Evaluation Team should review the packet for content prior to arriving on site. The Evaluation Team Leader may ask for clarification or additional information after receiving the application package from Administration. If you, as an evaluation team member, need additional information for review prior to the visit, please request this through your evaluation team leader, who will compile this information and inform the program entity of what is needed.

THE ACCREDITATION SITE VISIT Onsite visits for initial accreditation will typically be three days depending upon the size and complexity of the program. Reaccreditation onsite evaluations will be for two days unless there are extenuating circumstances. An agenda for the onsite evaluation should be arranged between the evaluation team leader and the program director (or other program official) representing the program seeking accreditation (see Model Program Evaluation Agenda, Appendix B). The model agenda outlines important events which should take place during the onsite evaluation. The agenda may be modified by the evaluation team leader in coordination with the host program due to local circumstances. This should be done well before the visit is to take place. The evaluation team leader should then furnish a copy of the agenda

https://ifsac.org/images/da_pdfs_docs/DA_Accreditation_Criteria.pdf

https://ifsac.org/images/da_pdfs_docs/DA_Accreditation_Criteria.pdf


to each member of the onsite evaluation team prior to arrival. Officials representing the program seeking accreditation should take part in the preparation of the agenda so that it accommodates the characteristics of local facilities and allows for scheduled interviews with appropriate faculty, students, and administrators. Arrival

Upon arrival, the evaluation team leader will contact the program representative for any final modifications to the schedule. After arrival and check-in, site team members should have a private meeting to discuss any strategies or assignments for the onsite evaluation. The schedule should be reviewed at this time, as well. Site Team Work Area

The onsite evaluation team should be provided a private work area (e.g., conference room) where they can discuss issues without interruption. If possible, the room should also have a telephone and computer connected to the internet. Preliminary Meeting with Program Staff

Following an opening conference with program officials to state the purpose of the evaluation, and onsite evaluation team expectations and needs, it is acceptable for the individual onsite team members to undertake separate interviews and visits within the program. Onsite team members should plan to come together for conferences and interviews as necessary. Visit and Review of Support Services

The team should take the time to review any support services that may be relied upon by the program to carry out their mission. Onsite team members should feel comfortable that these services provide the support indicated by the program. Staff Interviews

During an onsite evaluation, onsite evaluation team members should interview agency personnel, such as the director or program chair, department staff and/or faculty, the academic dean, commission or advisory committee, and students. Program Director

The individual responsible for the overall program should be interviewed separately from staff. Onsite evaluation team members should watch for items such as consistency with other employee views, compatibility with written procedures, and a thorough understanding of the program. Faculty The onsite evaluation team members should interview enough instructors to ensure


overall program understanding and commitment to written policies and procedures, as well as consistency with each other and the program director. Advisory Board The onsite evaluation team members should attend an advisory board meeting or meet with advisory board members to ensure that the advisory committee is representative of the population served and that they have input in program planning. Student Interviews Probably the most effective means of finding out when procedures and policies may not be consistent is through interviews with students participating in the program. Interviews with students should be conducted without the presence of representatives of the program seeking accreditation. Interviews should be short, and care should be taken not to pressure them, or interrupt the student's academic requirements. They should be made to understand that their interview has nothing to do with their individual performance and how critical their information is to the overall quality of the program. Conducting Interviews

To gain the maximum utilization of time during the visit, each onsite evaluation team member should have a strong familiarization with interview tactics. This will help ensure an efficient interview and information-gathering session, as well as present a positive image of IFSAC to the staff and professionals of the program. Following are some tips for conducting successful interviews: • Punctual - Be on time. If individuals have been scheduled to appear before the onsite

evaluation team, they may be nervous. Keeping them waiting is both discourteous and may increase their nervous tension. Professionalism is increased by being on time.

• Adaptable - Be flexible in manner, adjusting to different personalities of the people being interviewed. Be prepared to change the scheduled or planned questions based on new areas that may present themselves in the interview. Expand on areas that appear to be strong in the individual.

• Courteous - These people are not on trial. Treat them with professional courtesy, and remember that they are helping you, and may be doing this voluntarily.

• Personable - Be well-groomed, neat, and friendly. Don't look like a police investigator after a criminal. Allow the environment to be relaxed.

• Poised - Regardless of what information comes out in the interview, remain poised and direct. Don't act surprised by information, and don't allow or force the individual to feel that he or she has disclosed some unknown secret.

• Persistent - Be persistent without being aggravating. Continue on a point, short of becoming overbearing, to make sure you have obtained complete information.


• Notes - Take good notes, and take them continuously. Don't jump to your pad on a particular issue, making them feel they just "spilled the beans." Good notes allow for a good report, and make documentation for the report much easier.

• Attitude - Be positive. You're not there to convince the people that you know more than they do. Remain positive about their program, even in light of negative information.

• A good listener - Listen. Don't lead the individual. Remember that they are providing you with information, not vise-versa.

• Unbiased - Remain unbiased on issues with which you don't agree. Consider before a site visit begins that there may be some items about a program you do not like. Keep your opinions to yourself and judge things on the merit of the accreditation criteria.

• Concerned - Present the impression that you are concerned and appreciative of the information being provided, that you have a deep desire to help improve the program, and that you care about it.

• Discerning - You have to evaluate the reliability of what is being told. Mentally compare the information being voiced to you in comparison to written procedures.

• Knowledgeable - Allow the individual being interviewed to recognize that you know the subject as well. This will build a bond and encourage honesty.

• Impressive - Make a good first impression, as well as a good closing impression. In closing, thank each individual for his or her time.

• Fair - Be fair in your questions, recognizing that some questions are fair to ask certain individuals, and others are not. Allow the individual to express items without a question, such as closing remarks. This will allow the individual to feel as if he or she has contributed without request, allowing for some surprise information.

• Be Yourself - More than likely, you perform a similar job in your program. Don't present the impression that you feel high and mighty. Someone may be interviewing you for the same reason someday.

The Exit Conference

The agenda should include a private onsite evaluation team meeting before the exit conference to reach consensus on findings, to continue preparation of the final report, and to designate onsite evaluation team member roles for the exit conference. The agenda should also indicate prompt closure at the end of the exit conference, with the immediate departure of the team. A final exit conference takes place between the onsite evaluation team and the program director, as well as any other principles the program director wishes to involve. This is the most challenging segment of the onsite evaluation during which the onsite evaluation team explains to the program director exactly what they have found. To prepare for the exit conference, onsite evaluation members should review their findings with other


members of the evaluation team. Make sure all essential information is available to make a decision regarding each criterion in the Criteria for the Accreditation of Degree Programs of the IFSAC Degree Assembly. Each NO statement on the checklist should be accompanied by a recommendation or requirement on a Final Action Report form (Appendix D) and a rationale for the recommendation or requirement. Decide whether any suggestions related to "weak" areas in the program will be made. Decide how the onsite evaluation members will divide the information to be conveyed in the exit conference (who will say what). The evaluation team leader begins the exit conference by expressing appreciation for the hospitality extended and the arrangements made by the program director and other representatives of the host program. The evaluation team leader then reviews the manner in which the exit conference will take place. The evaluation team leader explains that the purpose of the onsite evaluation was to assess the program's compliance with the Criteria for the Accreditation of Degree Programs of the IFSAC Degree Assembly at the time of the evaluation and that the evaluation team will now review its findings. The evaluation team leader should begin the discussion by listing the strengths of the program. Identifying strengths as well as weaknesses provides the host program with a more balanced report. This should be followed by a discussion of the specific areas related to the Criteria for the Accreditation of Degree Programs of the IFSAC Degree Assembly in which the program is weak, and how to improve in these areas. The host program should be informed in detail of all findings before the onsite evaluation team leaves the site. There should be no surprises when the program seeking accreditation receives the final report. During the exit conference, the program director and others should be allowed to ask any questions and/or make any comments that they wish. However, the discussion should avoid debating the accreditation criteria or the evaluation team's findings. The representatives of the host program should have the opportunity to understand why any deficiencies were cited. The host program should also have an opportunity to clarify the findings at this point. In closing the exit conference, the evaluation team leader should explain that the evaluation visit team members are fact-finders only and that the Board will make the final decision on accreditation status. The evaluation team leader should also explain the next steps in the accreditation process and any corrective actions which remain at the end of the evaluation. The evaluation team leader then thanks the host once again and lets them know that the evaluation team's responsibilities are over and if they have any further questions regarding the evaluation, they should contact Administration. Deficiencies, Recommendations, and Requirements

The evaluation site team leader will make the determination of when to disclose any


deficiencies, recommendations, or requirements that are discovered during the reading of the self-study or during the onsite evaluation to the program requesting the program evaluation. The following shall serve as guidelines of when and how the deficiencies, recommendations, or requirements are revealed and recorded. 1. If there are major deficiencies found during the reading of the self-study, the onsite

evaluation will be postponed and the program will be asked to provide a more complete self-study. The program will be advised that it is highly recommended that they work with a mentor that has previously completed a successful self-study to assist with the self-study revision.

2. If there are criteria found during the reading of the self-study that need further explanation prior to the onsite evaluation, the evaluation team leader will contact the program entity and copy Administration to request the additional information in writing from the program.

3. If perceived deficiencies, recommendations, or requirements were due to unclear or incomplete information in the self-study, but it is found that the program is actually meeting the criteria, then these items will not be noted in the program evaluation report or on a Final Action Report form.

4. If deficiencies, recommendations, or requirements are found during the reading of the self-study or during the onsite evaluation, but are corrected before the end of the onsite evaluation, they still should be noted in the program evaluation report and on final action reports as appropriate.

Review of Report Forms

Each onsite evaluation member must approve and sign the report form. It is suggested that prior to leaving the entity’s site, the onsite evaluation team should complete a first draft of the report. .After this, the evaluation team leader can return home and organize the report to its final form. The report should then be mailed/faxed to the remaining onsite evaluation team members for concurrence. The final program evaluation report is to include: 1. Narrative report. 2. Final action reports for any recommendations or requirements noted in the narrative

report. 3. Completed Institution Self-Study and Evaluator Checklist and verification of the

items listed on the last page of this checklist. 4. Supporting documentation as appropriate.

https://ifsac.org/images/da_pdfs_docs/Inst_SS__Evaluator_Checklist.docx


AFTER THE SITE VISIT The evaluation team leader is responsible for the completion and filing of the team report in draft form with Administration within thirty (30) days of the site visit. Before endorsing it by signature, each onsite evaluation team member must review the final report to ascertain the following: 1. It is legible, clear and accurate, without important omissions. 2. Personal or unverified observations have been removed. 3. Editorial improvements have been made as necessary. 4. Deficiencies cited are supported in the body of the report, each deficiency references

one or more specific criteria for degree accreditation, and the deficiencies (recommendation or requirement) are noted on a Final Action Report form.

Administration is responsible for forwarding the report along with any comments that were received from the program seeking accreditation to the DABOG thirty (30) days prior to the next scheduled meeting of the DABOG. The report and any correspondence received will be reviewed by the DABOG, who will then make a decision. The program will then receive notice of the DABOG’s decision in writing within thirty (30) days after the meeting.


APPENDICES Appendix A IFSAC Degree Assembly Procedures for Scheduling a

Program Evaluation Appendix B Model Program Evaluation Agenda

Appendix C Model Program Evaluation Report

Appendix D Final Action Report

Appendix E Degree Program Evaluator Selection Procedure

Appendix F Timeline

Endnotes


APPENDIX A - IFSAC DEGREE ASSEMBLY PROCEDURES FOR

SCHEDULING A PROGRAM EVALUATION


IFSAC Degree Assembly Procedures for Scheduling an Initial or Reaccreditation Program

Evaluation

Note: The following process will take a minimum of ninety (90) days after a completed self-study is received and must take place while school is in session.

Institution Program Responsibilities:

Step 1: The institution receives, from ifsac.org website, an Application for Accreditation and Institution Self-Study and Evaluator Checklist.

Step 2: The institution returns the completed Application for Accreditation to the IFSAC Administrative Office. When an institution is requesting accreditation for more than one degree program, it MUST list all degrees to be reviewed on the application.

Step 3: The institution conducts the self-study using the IFSAC Institution Self-Study and Evaluator Checklist. The institution must complete and return, to IFSAC Administrative Office, the self- study at least 90 workdays prior to the requested site visit start date.

Step 4: The institution prepares additional copies of the completed self-study, including all supporting documentation. Once Administration confirms the evaluation team, the institution seeking accreditation will send the additional copies to each onsite evaluation team member (2 or 3, as applicable) and reader (2).

• The self-study may be submitted in print or electronic format.

• Electronic submissions must be by CD, DVD, thumb drive/jump drive, or email.

• If submitted by email, all files should be in one compressed (zipped) folder or a PDF portfolio so that all items download at once. [Note: File servers of the recipient may prohibit large file downloads.]

• Submission via “cloud” platforms or Dropbox or sources will not be accepted.

IFSAC Responsibilities:

Step 1: Upon receipt of the completed application and application fee (may be invoiced) for accreditation from the institution, IFSAC staff shall verify the degree(s) to be accredited and the dates for the site visit.

Step 2: IFSAC staff will update the institution file.

Step 3: IFSAC staff will initiate the procedures for selection of evaluators. Final confirmation of onsite evaluation team members will not be completed until the application for accreditation and self-study documents are received by Administration.

https://ifsac.org/images/da_pdfs_docs/Application_for_Accreditation.docx

https://ifsac.org/images/da_pdfs_docs/Application_for_Accreditation.docx





APPENDIX B - MODEL PROGRAM EVALUATION AGENDA


IFSAC Onsite Team Schedule Community College Fire Science Technology, A.A.S. Degree

March 21, 2011 – March 25, 2019

Monday, March 21, 2019 2:00 PM Evaluation team member #1 arrives at airport 2:40 PM Evaluation team member #2 arrives at airport 2:55 PM Evaluation team member #3 arrives at airport Onsite team members check into hotel and prepare for onsite evaluation. Tuesday, March 22, 2019 8:10 AM: Opening Interview with College representatives (President, Dean,

Department Staff, other college officials decided upon by Onsite Team Leader and hosting program).

9:10 – 11:30 AM Evaluate Classes – Meet instructors – Interview Students / Review

Documentation 12:00 – 1:30 PM Lunch 1:30 PM Review Library to examine holdings 2:30 PM Meet with Student Service Representatives 3:30 – 5:00 PM Work documentation and discuss criteria/self-study Wednesday, March 23, 2019 8:00 – 10:00 AM Observe Fire Science Advisory Committee – Meet with committee

members 10:00 – 11:30 AM Meet with Dean and other Staff Members 12:00 – 1:00 PM Lunch 1:00 – 5:00 PM Work on report / discuss findings / meet with college officials as needed Thursday, March 24, 2019 8:00 – 11:30 AM Meet with College Officials as needed / Complete Evaluation Report /

Complete Final Action Reports as needed and prepare for exit interview 12:00 – 1:00 PM Lunch 2:00 Exit Conference with College Officials Onsite Team Members leave late afternoon or following morning.


APPENDIX C - MODEL PROGRAM EVALUATION REPORT


ABC College Fire Science Technology Associate in Science Degree Program

Accreditation Evaluation Report International Fire Service Accreditation

Congress March 3-5, 2019

International Fire Service Accreditation Congress (IFSAC) Team: Visiting team: Name, Institution Represented Name, Institution Represented Name, Institution Represented Readers: Name, Institution Represented Name, Institution Represented The onsite evaluation team arrived in location on Sunday, March 2 and over the next three days conducted an onsite evaluation review of the ABC College Fire Science Program. The ABC College is a comprehensive community college and is part of the College District. ABC College has an average semester enrollment of 22,000 credit students and is the largest single-campus community college in Texas and one of the largest in the United States. The Fire Science Program, in addition to the EMS and Criminal Justice programs, fall under the Protective Services Department. Fire Science offers certificate training and three A.A.S degrees. The Fire Science Department is seeking reaccreditation for an A.A.S. Degree in Fire Science and an A.A.S. Degree in Fire Arson Investigation. The self-study was exemplary and our program evaluation only confirmed the details outlined in the self-study. After the completion of the onsite evaluation we found all requirements of the “Criteria for Accreditation” to be met and offered four recommendations to further strengthen the program during our exit interview (see attached). The following is a summary of our evaluation: Monday, March 3 We met with Dr. Zeigler, President of ABC College. Dr. Zeigler welcomed us to the college and offered to assist us with our review in any way possible. He confirmed that he, along with the rest of the college administration, is very supportive of the career and technical programs at the college including the Fire Science Program. Dr. Zeigler discussed the future plans/vision of building a First Responder training center off- campus that would house the Fire Science Program further confirming the college’s long- term commitment


to the Fire Science Program. The onsite evaluation team met with Program Director on Monday morning to discuss program specifics, obtain clarification on some of the self-study documentation, and better understand the programs offered by the Fire Science Department. The Program Director was very helpful and informative regarding the program and were great hosts throughout the evaluation process. Jane Smith is the administrative support person for the Protective Services Department. Jane showed Evaluation Team Member the student and faculty records, the storage area, and audio visual equipment used by the department. The Evaluation Team Member was very impressed with the record keeping system establish by Jane and the college’s procedure used to double check graduation applications. The Fire Administration II class met on Monday afternoon. The onsite evaluation team audited the class for a short time and also spoke with some of the students. Some of the students were attending class to acquire learning credits for state certification requirements while others were taking the class as a requirement of the degree. The class was made up of a diverse group of students including a captain from a career department, students from volunteer departments, and students pursuing the fire service as a career. The remainder of Monday was spent visiting different areas of the college to include the library, office area, student advising area (confidential area), and student center. We also reviewed syllabi, master course documents, and transcripts of graduates to verify students are meeting program requirements. Tuesday, March 04 Tuesday started with a meeting with the VA Benefits advisor, Mr. Demasi. Mr. Demasi explained how he works with Fire Science students that qualify for VA Benefits and that want to pursue the Fire Science Program. Mr. Demasi stated that he has a great working relationship with the Fire Science area and was very complementary of the program director. Following our discussion with Mr. Demasi, we met with Mr. Singh from the Library. Mr. Singh demonstrated how students affiliated with ABC College can access thousands of documents online. Mr. Singh also confirmed that the Fire Science Program has direct input in resource purchases for the library and have a $10,000 budget for such purchases. The rest of Tuesday morning was spent visiting Fire Academy locations, reviewing the equipment, academy file locations, and auditing a fire academy class. We also observed two counseling sessions of fire academy students and Evaluation Team Member audited part of a fire academy lecture. The equipment, supplies, and classrooms were all more than adequate to meet the needs of the program. Tuesday afternoon started with a meeting with Dr. Mendiola, Interim Dean of Student


Affairs. Dean Mendiola discussed the role of Student Affairs and the student discipline process. Dean Mendiola also mentioned how impressed she was of the Fire Science Program. Evaluation Team Member met with Mr. Campos from the Financial Aid Office. Mr. Campos reviewed the financial aid process, grants and scholarships available, and how ABC College’s financial aid requirements are more stringent than most colleges in the state. Approximately 45% of the student body is on financial aid. Mr. Campos provided Evaluation Team Member with a copy of the financial aid process that he put together for the college department chairs for quick reference. Mr. Campos also confirmed that student records are maintained in locked files. Mr. Campos, as most others on the campus that we spoke with, complemented the Fire Science Program. Tuesday afternoon Evaluation Team Member audited the Tactics and Strategy class and met with one of the students. The student is finishing his degree this semester. The student confirmed that course content material is very applicable to the duties expected in the fire service. Wednesday, March 5 Wednesday morning was spent completing the evaluation team reports and meeting with the administration for the exit interview. The evaluation team unanimously recommends reaccreditation of ABC College for the two degrees requested. Respectfully submitted, Evaluation Team Leader Date Evaluation Team Member Date Evaluation Team Member Date


APPENDIX D - FINAL ACTION REPORT


International Fire Service Accreditation Congress FINAL ACTION REPORT

Please select one:

Requirement Recommendation

Name of Institution

Degree Program involved

EVALUATION AREA

CONDITION NUMBER

CONDITION

Format requirements

Evidence that condition has been met should be sent to:

By the following date

If evidence that condition has been met cannot be completed by due date, contact:


APPENDIX E - DEGREE PROGRAM EVALUATOR SELECTION

PROCEDURE


International Fire Service Accreditation Congress Degree Program Evaluator Selection Procedure

PURPOSE

The purpose of this procedure is to outline the qualifications, selection criteria, and procedure for selecting Degree Program Evaluators.

QUALIFICATIONS Degree Program Evaluation Team Member Qualifications

An evaluation team member shall be a member Emeritus or any person appointed by the DABOG who meets the following qualifications:

• Possess a degree - preferably in a fire-related training or educational area.

• Have attended degree program evaluator training within the last three years, or after initial training, a team member must have attended refresher training every three years.

Exceptions to Training Qualifications for Team Members:

Until such time that refresher training becomes available by a distance learning method, Emeritus members and past/present public members who have served as a site team member or site team leader at least three times in a three-year period may be exempted from the refresher training requirements for a like number of years.

Administration will maintain a list of qualified evaluators.

Degree Program Evaluation Team Leader Qualifications

The team leader shall possess the qualifications and experiences listed above and shall have:

• Served as a degree program evaluation team member.

• Successfully completed degree program evaluation team leader training within the last three years; or after initial training, a team leader must have attended refresher training every three years.

SELECTION CRITERIA

Degree program evaluators shall:

• Not act as a compensated consultant to the institution in any manner pertaining to accreditation.

• Not be from the same state, province, or geographical boundary; (for example, college A is in Portland, Oregon and college B is College A’s competitor right across the border in Vancouver, WA.).

• Not be a graduate or past or present employee of the institution seeking


accreditation.

• Not have received any compensation from the institution seeking accreditation.

• Possess peer review capability - in terms of having relevant experience of appropriate programs, student population, etc. (for example, evaluators must have experience of program length and diversity of the type they will be evaluating).

SELECTION PROCEDURE

Step 1: Within 30 days following annual Degree Assembly site team training, Administration shall compile a list of people qualified to participate in the accreditation evaluation process for the next year (annual training to next annual training). The list shall be compiled into three qualification categories (site team leader, site team experienced member, and site team member without experience) in priority order based on the person’s last site team participation (not including reader).

Step 2: Administration reviews the list of qualified degree program evaluators, and selects five (5) qualified evaluators, and two (2) qualified team leaders.

During reaccreditation site visits, the members of the last site team shall be ineligible for the current visit unless circumstances mandate a prior visitor to be a member of the team.

Leaders and visitors should be rotated from the top to the bottom of the list (prioritized) based on visits completed. Administration should attempt to include one qualified member on each visit who has not attended a visit previously.

Step 3: Readers will be designated as alternate members of the site team. One of the selected team leaders shall be designated as the primary team leader and the other as the alternate. If a site team member/leader must cancel because of an emergency, Administration will notify the appropriate alternate as soon as possible. The program will make travel arrangements for the alternate.

Step 4: Administration indicates the names of prospective team leaders, site visit team members, and readers on the list and places these names in priority order from one (1) through five (5) (#1 is first priority).

Step 5: Administration should provide the program representative with the list compiled for approval at least 75 days prior to the requested site visit dates. Evaluators on the list shall be placed on the list in priority order, including two site team leaders and five site team members/readers, and copies made for recording purposes.


Step 6: The program representative reviews the list, strikes any names that are unacceptable, and returns the approved list within 10 days to Administration.

Step 7: Administration contacts each member on the list, and the alternate, to verify availability based on site visit dates identified on the program's application.

Step 8: Administration determines who will be the team leader, team members, and readers.

Step 9: Administration sends a confirmation letter containing program contact information, pertinent site visit information, evaluation team leader/members/reader contact information, etc., to program representative, site visitors, and readers.**In the case of a reaccreditation where only two members will attend the site visit, Administration shall use the same process outlined in the steps above.

The team member may be experienced or inexperienced based on availability.


APPENDIX F - TIMELINE


IFSAC Degree Assembly

Site Visit Timeline for Accreditation/Reaccreditation

• Program requesting accreditation reviews accreditation documents located on

IFSAC’s webpage, including the application for accreditation and the institution self-study and evaluator checklist.

Minimum of 90 Days Prior to Site Visit Date Requested

• Program submits application for accreditation, completed Institution Self-Study and Evaluator Checklist, and payment to Administration.

• Administration verifies the following: o Program is a voting member (Bylaw 21.5.1(A)(1) & 22.5.2) o Program has at least one graduate from the program (Bylaw 22.5.6) o Verify the degree(s) to be accredited (Procedures for Scheduling a Site

Visit) o Dates for the site visit (Procedures for Scheduling a Site Visit) o Program has regional accreditation or state sanction (Bylaw 22.5.4) o Self-study meets initial review for completeness (Bylaw 22.5.5)

Minimum of 60 Days Prior to Site Visit

• Onsite evaluators & readers selected per the Degree Program Evaluator Selection Procedure.

• Program forwards contact information to Administration.

• Administration forwards contact information for onsite evaluation team members, readers, and the program contact person to each of the aforementioned.

Minimum of 45 Days Prior to Site Visit

• Program confirms with the evaluation team leader that key institution personnel are available to meet during the onsite evaluation.

• Evaluation team leader and program develop onsite evaluation agenda.

• Program delivers self-study document and supporting documents to program evaluators.




Minimum of 2 - 6 Weeks Prior to Site Visit • Travel, lodging, and other logistical arrangements completed (Program

Evaluation Information and Procedures Document – Preparing for the Accreditation Site Visit).

Maximum of 30 days After Site Visit

• Final narrative report, final action reports, and verification of all the items on the last page of the Program Evaluator Checklist (Bylaw 22.5.7)

Maximum of 45 days After Site Visit • Institution provides appropriate reimbursement to onsite evaluation team

members.


ENDNOTES

1 Adapted from the American Medical Association’s Accreditation Manual, Committee on Allied Health Education and Accreditation, 1991.

2 Portions adapted from the “Site Visitors in the Accreditation Process: A Guide

to Issues and Practical Concerns, Council on Postsecondary Education, October 1988.

3 Adapted from the American Medical Association’s Accreditation Manual,

Committee on Allied Health Education and Accreditation, 1991.

4 Adapted from the American Medical Association’s Accreditation Manual, Committee on Allied Health Education and Accreditation, 1991.

5 Adapted from "A Decalogue for the Accreditation Team," Hector Lee (COPA

Agenda, February 5, 1976 and the American Medical Association Accreditation Manual from the Committee on Allied Health Education and Accreditation.

Documents

Program Evaluation Information and Procedures · program evaluation team member with the name, address, and phone number of a representative of the host programwho serves as the local