KPI Student Satisfaction and Engagement Survey 2015 · PDF fileKPI Student Satisfaction and Engagement Survey 2015-16 Survey Cycle Ministry of Training, Colleges and Universities

KPI Student Satisfaction and Engagement Survey

2015-16 Survey Cycle

Ministry of Training, Colleges and Universities

Colleges of Applied Arts and Technology Policy Framework

1.0 Governance and Accountability

Operating Procedure



Issued: April 1, 2003 Revised: May 1, 2015






Operating Procedure

Issued: April 1, 2003 Revised: May 1, 2015 Page i

TABLE OF CONTENTS

Purpose and Application ................................................................................................. 1

Glossary .......................................................................................................................... 2

Requirements .................................................................................................................. 3

Key College Contact .................................................................................................... 3

Advanced Survey Notification ...................................................................................... 5

College-Specific Questions for the Advanced and Formal Surveys ............................. 5

Postsecondary Programs of Instruction File .......................................................... 6

Packaging File ............................................................................................................. 9

Local Agreement........................................................................................................ 10

Student Profile Data File ............................................................................................ 12

Communication .......................................................................................................... 13

Survey Protocol ......................................................................................................... 13

Student Survey Population ..................................................................................... 14

Survey Administration Procedures ......................................................................... 17

Return of Completed Surveys ................................................................................ 20

Distribution of Survey Results and Reports ............................................................... 20

Release of Key Performance Indicator Data .............................................................. 21

Review and Adjustment of the Student Satisfaction KPI ............................................ 22

Further Information ........................................................................................................ 22

Summary of Responsibilities ......................................................................................... 22






Operating Procedure

Issued: April 1, 2003 Revised: May 1, 2015 Page ii

Colleges of Applied Arts and Technology .................................................................. 22

Ministry of Training, Colleges and Universities .......................................................... 23

Appendix A: Key Dates ............................................................................................... 24

Appendix B: Supplier's Contact Information .................................................................. 31

Appendix C: Student Satisfaction and Engagement Survey Instructions for Students ....................................................................................................................... 32

Appendix D: Packaging File .......................................................................................... 34

Appendix E: Student Profile Data File by Program .................................................. 35

Appendix F: Instructions for Survey Administrators ....................................................... 37

Appendix G: Process Verification .................................................................................. 40

Appendix H: Student Satisfaction and Engagement Survey ................................... 41

Appendix I: Survey Results and Reports ....................................................................... 46

Appendix J: Review and Adjustment of the KPI Data .................................................... 49

Appendix K: College Codes ....................................................................................... 51

Appendix L: Non-traditional Pattern of Program Delivery .............................................. 52

Appendix M: Surveying Apprentices in 2015-16 ..................................................... 53






Operating Procedure

Issued: April 1, 2003 Revised: May 1, 2015 Page 1

Purpose and Application

Effective governance and appropriate accountability mechanisms are key factors in ensuring that the citizens of Ontario are well served by government agencies and transfer payment recipients. They are also crucial to the successful achievement of the colleges’ objectives. Performance measurement through the collection of performance indicators is an important accountability tool.

The Ministry of Training, Colleges and Universities (MTCU) and the colleges of applied arts and technology have defined five Key Performance Indicators (KPIs) to measure, in a consistent manner across the college system, college performance against Ministry stated goals and objectives. The five KPIs are: graduate employment;

graduate satisfaction;

employer satisfaction;

student satisfaction; and

graduation rate.

The first three KPIs – graduate employment, graduate satisfaction, and employer satisfaction – are used to distribute performance funding to the colleges.

The accountability requirements for colleges are outlined in the Minister’s Binding Policy Directive on Governance and Accountability Framework that the Minister of Training, Colleges and Universities has issued pursuant to the Ontario Colleges of Applied Arts and Technology Act, 2002. The Act also authorizes the Minister to request any financial or other information from the colleges.

This operating procedure, which applies to all colleges, sets out the requirements for the colleges’ administration of the Student Satisfaction and Engagement Survey used to collect the Student Satisfaction KPI data, and for the publication of the KPI data. Standard survey administration procedures are used to ensure a consistent, objective approach across all colleges, and to yield survey results that are reliable, comparable, and verifiable. The intent of the survey protocol is to safeguard the integrity of the survey results by administering the survey in an impartial manner and in a controlled environment.

http://caat.edu.gov.on.ca/

http://www.ontario.ca/laws/statute/02o08f






Operating Procedure


These procedures were developed by the joint College-Ministry KPI Steering Committee that provides advice and makes recommendations to the Ministry on the administration of the KPI initiative.

The KPI data are used by colleges to demonstrate their achievements and to identify where changes could be made to programs and services to better meet the needs of students. The Ministry also uses the data to advise and inform government about the colleges and in the planning and policy-making for the college system.

Glossary

Advanced surveys: the Student Satisfaction and Engagement Survey that colleges may choose to administer during a specified two-week period in June to those approved through the Non-Traditional Pattern of Program Delivery option and in November each year to field, co-op, and clinical placement students who are unavailable to be surveyed in February, or to those approved through the Non-Traditional Pattern of Program Delivery option.

Co-op Diploma Apprenticeship (CODA): joint program of instruction that includes all required apprenticeship in-school training; diploma course requirements; and at least one on-the-job paid work placement in the relevant trade, which contributes required apprenticeship hours towards the participant’s Certificate of Qualification.

Formal survey: the Student Satisfaction and Engagement Survey that colleges are required to administer to the student survey population during a specified two-week period in February each year.

Postsecondary program of instruction: collectively, basic postsecondary and post-basic programs of instruction, which are defined below.

Basic postsecondary program: a program of instruction that conforms to thelevels of learning articulated in the Credentials Framework (see Minister’sBinding Policy Directive on Framework for Programs of Instruction: Appendix A)and leads to the awarding of one of the following credentials: Ontario CollegeCertificate, Ontario College Diploma, Ontario College Advanced Diploma, or anapplied degree. Basic postsecondary programs also include joint college-university programs that lead to the awarding of a degree by the universitypartner.

Post-basic program: a program of instruction that conforms to the levels oflearning articulated in the Credentials Framework (see Minister’s Binding Policy







Operating Procedure


Directive on Framework for Programs of Instruction: Appendix A) and leads to the awarding of an Ontario College Graduate Certificate.

Student governments(s): the minimum number of student organizations that have both elected leadership and by-laws setting out their methods of operation, are recognized by the college’s board of governors, and, when viewed in combination at an individual college, represent all students [belonging to the survey population] who are eligible to vote for a student organization (definition as stated in the operating procedure Tuition and Ancillary Fees Reporting, Appendix C, Requirements for Protocols for Introducing or Increasing Compulsory Non-Tuition-Related Incidental Fees).

Student Satisfaction KPI: the average percentage of students who completed the survey and who responded that they were satisfied or very satisfied with the quality of their learning experience, support services, and educational resources.

Student survey population: all students enrolled in postsecondary programs of instruction approved for funding through the College Funding Framework (CFF), or enrolled in CODA programs of instruction approved for funding through CODA contracts or enrolled in apprenticeship in-class training, including those enrolled in applied degree programs, except for:

Requirements

These mandatory requirements are for the 2015-16 survey cycle, detailed in Appendix A.

Key College Contact

Each college is to designate a staff person to serve as a Key College Contact for the Student Satisfaction and Engagement Survey to work with the Ministry and the Supplier

Students who are:

registered in their first semester of their program of instruction; or

have been given advanced standing and are in the program of instruction for the first time.

Students who are registered as part-time students and are NOT pursuing a certificate, diploma or an applied degree.

http://caat.edu.gov.on.ca/PolicyFramework/FrameworkforPrograms.pdf







Operating Procedure


contracted by the Ministry to refine the survey instrument, prepare and deliver the survey packages to the colleges, and tabulate and report the KPI results.

An alternative staff member is to be named and is expected to carry out the responsibilities detailed below when the Key College Contact is not available.

The Key College Contact is to:

Ensure that the college informs the Ministry whether or not the college willconduct the advanced survey in June and/or November.

Provide the Supplier up to five college-specific questions through the onlinesystem.

Provide the Supplier with a list of college postsecondary programs of instructionfunded by the Ministry through the general purpose operating grant through theonline system.

Provide the Supplier with the packaging file and student profile data file of thecollege through the online system.

Work with the student government(s) to identify its role in the StudentSatisfaction and Engagement Survey process and develop a local agreementformalizing the college’s survey process.

Work with the Supplier on college-specific procedures for the delivery and returnof the surveys.

Communicate and promote the Student Satisfaction and Engagement Survey tocollege students, faculty, support staff, management, and student government(s).

Coordinate the administration of the Student Satisfaction and EngagementSurvey.

Ensure that the college meets the deadlines as outlined in this operatingprocedure.






Operating Procedure


Advanced Survey Notification

There are three periods in the academic year during which the StudentSatisfaction and Engagement Survey is conducted, including: two weeks in June,which is optional; two weeks in November, which is optional; and

two weeks in February, which is mandatory.

Only students approved through the Non-Traditional Pattern of Program Delivery option may participate in the advanced survey in June.

Only students who will be on field, co-op, or clinical placements during the February survey period may participate in the advanced November survey, or those approved through the Non-Traditional Pattern of Program Delivery option.

A student may not be surveyed twice during a survey cycle.

The Key College Contact is to ensure that the college e-mails the Ministry in April indicating whether or not the college will be undertaking the advanced survey in June. No changes can be made after this date. The letter is to be e-mailed to the Postsecondary Accountability Branch director at [email protected] The Key College Contact is to ensure that the college e-mails the Ministry in August indicating whether or not the college will be undertaking the advanced survey in November. No changes can be made after this date. The letter is to be e-mailed to the Postsecondary Accountability Branch director at [email protected].

Key dates to notify the Ministry about conducting the surveys and the schedules for each survey period are shown in Appendix A.

College-Specific Questions for the Advanced and Formal Surveys

Colleges may submit to the Supplier up to five college-specific questions to be added at the end of their Student Satisfaction and Engagement Survey. The questions and the responses are not disclosed to the Ministry and they are not included in the calculation of the KPI.

Colleges are to adhere to the following guidelines

Use the online system to submit the college-specific questions.

mailto:[email protected]







Operating Procedure


Provide questions in the language (i.e., English or French) in which the surveysare to be conducted.

Structure the questions as closed-ended questions and provide a singleresponse list. Each response opportunity counts as one question, for example, afive-part question counts as five questions.

Indicate your agreement to share the questions with other colleges who maywant to view questions from other colleges.

The Key College Contact is to send via the online system the five college-specific questions for the June advanced survey to the Supplier in April (see Appendix B). The same questions will be used for the November advanced survey and the formal survey in February. Colleges cannot submit an additional five college-specific questions for the November or formal survey.

Colleges that did not participate in the June advanced survey but are participating in the November advanced survey are to send via the online system the college-specific questions for the November advanced survey to the Supplier in September (see Appendix B for the Supplier's address). The same questions will be used for the formal survey in February. Colleges cannot submit an additional five college-specific questions for the formal survey.

Colleges that participate only in the formal survey are to provide their college-specific questions to the Supplier in November through the online system.

The Key College Contact is to verify that the college-specific questions prepared and sent by the Supplier are correct.

The deadline dates for submission and verification of the college-specific questions for the advanced and formal surveys are shown in Appendix A.

Postsecondary Programs of Instruction File

Advanced June, Advanced November and Formal Surveys

Colleges will provide a list of postsecondary programs of instruction through the online system. The file may include only a subset of the college programs for the first advanced survey in which they participate. For each successive survey the college may add more programs. These will be appended to the programs already provided to the






Operating Procedure


Supplier. Programs provided in previous surveys (i.e. one of the advanced surveys) cannot be changed or removed.

The colleges that do not participate in either the advanced June or November survey are to prepare and submit their postsecondary program of instruction file in November using the same procedures.

The online system checks the file format automatically and displays any errors in the file. If the file is correct, the Key College Contact is advised and given the opportunity to send the file to the Supplier. If the file contains errors, the Key College Contact is to ensure that the appropriate revisions are made to the file and then resubmit the file in the same manner. Some errors will be flagged as critical and must be corrected before the file will upload successfully. Other errors will result in a warning message only, that will not prohibit the file from being uploaded, but will allow the Key College Contact to make changes if deemed necessary.

The programs of instruction file includes the following and is to be sent by the due dates outlined in Appendix A:

A list of all college postsecondary programs of instruction approved for fundingby the Ministry1 by campus in an ASCII file for the instruction sheet for students(see Appendix C). Colleges that provide the list of programs for the advancedJune survey may provide additional programs for the November and then againthe Formal-February survey. However, they may not change nor removeprograms previously provided.

The program names should be names that students can easily recognize as theirprogram. Both program and campus names supplied in this file will be the namesdisplayed on college reports provided by the Supplier. Colleges should reviewthese program and campus naming conventions to ensure any truncation or useof proper (mixed) case titles are appropriate, prior to submitting the list to theSupplier.

The college campus postal code refers to the physical location of the campus.

1 These are the program groupings by which internal college reports can be prepared following receipt of all of the student satisfaction and engagement survey results in March each year.






Operating Procedure


The college may provide their own -digit program codes. When collegesprovide their own - digit codes, the first digit must be an alphabetical character between A and Z; the second digit must be numeric character between 0 and 9.

FIELDS Length (characters)

Position

1 College Code 4 1-4

2 Campus or location (e.g., Newnham) 30 5–34

3 Campus code (i.e., short form or abbreviation of campus or location, e.g., NM) 8 35–42

4 College program name (e.g., Law & Security) 45 43–87

5 College program name (long version for some reports) 75 88-162

6 College program code (i.e., short form or abbreviation of name code, e.g., LAW&SEC)

12 163-174

7 Approved program sequence (APS) number 5 175-179

8 MTCU code 5 180-184

9 College division/school/faculty name (optional) 45 185-229

10 College division/school/faculty code (optional) 8 230-237

11 -DIGIT alphanumeric program code (optional) 3 238-240

12 Campus location postal code (6 characters) 6 241-246

13 Apprenticeship (Y/N) 1 247

14 Class (numeric) 7 248-254

15 Trade (alpha-numeric – 4 or 5 characters) 5 255-259

THREE

Alternatively, the Supplier will generate a digit alphanumeric code for each program on the back of the instruction sheet. Students will be required to place this program code on the survey.

This list of college postsecondary programs of instruction must match the program names in the student profile data elements file that the college is to send to the Supplier in January. This one file will combine the student profile information from both advanced and formal surveys. If the Supplier generates the three-digit codes, then a file in a format identical to the format of the program list file with the three-digit codes will be available online for the college to use to produce the student profile data elements file.

The college code to be used in the postsecondary programs of instruction file is defined in Appendix K, and must be used in all files that the college is to send to the Supplier.

threethree

The third digit must be an alphabetical character between A and Z.

three-






Operating Procedure


16 Full-Time/Part Time (F/P) 1 260

17 Level (1, 2 or 3) 1 261

18 Session (1, 2, 3 or 4) 1 262

The Key College Contact will also be asked to provide:

The total number of surveys required for all survey periods, June, November andthe Formal Survey in February.

Approval that the college’s instruction sheet, prepared by the Supplier, is correct.

Packaging File

The Key College Contact is to send to the Supplier (see Appendix B for the Supplier's address), via the online system, the packaging file for each survey period independently.

The online system checks the file format automatically and displays any errors in the file. If the file is correct, the Key College Contact is advised and given the opportunity to send the file to the Supplier. If the file contains errors, the Key College Contact is to ensure that the appropriate revisions are made to the file and then resubmit the file in the same manner. Some errors will be flagged as critical and must be corrected before the file will upload successfully. Other errors will result in a warning message only, that will not prohibit the file from being uploaded, but will allow the Key College Contact to make a change if deemed necessary.

The packaging file includes the following and is to be sent by the due dates outlined in Appendix A:

An electronic list (see Appendix D) for each of the courses in which students areto be surveyed2.

2 Packaging file refers to college-specific information required for the Supplier to ensure the correct preparation of the survey packages for each college. The data elements in the file include the college name, campus, course/program name, course/program code, course/program section number, and number of students enrolled in the course/program section.






Operating Procedure


The college may include other identifiers to place on the pre-fixed label of each package to allow for the efficient delivery of survey packages to the appropriate class. Other identifiers may include the course section contact name, day of week, date, room number, and/or time.

To ensure correct sorting order, if the college opts to use the ‘Date’ field, the date should appear as yy/mm/dd. If ‘Day of the Week’ field is used, then a number must be assigned to each day of the week and that numeric must be the first character in the field, for example, 1-Mon, 2-Tue, 3-Wed, 4-Thur, 5-Fri, 6-Sat, and 7-Sun.

The Key College Contact will also be asked to provide via the online system:

The address (one per college) to which the courier is to deliver the box of surveypackage envelopes. Couriers do not deliver to post office box addresses. .Alternatively, colleges that wish to have survey packages delivered to individualcampuses may select this option online and provide addresses and sortinformation for each campus.

Instructions on how the survey packages are to be sorted (e.g., day of week andcourse name). If the college does not provide a preferred sort option to theSupplier, the default sort will be by course name and course code.

Approval of the verification report of the packaging file, including verification ofthe courier address to ensure that packages go to the Key College Contact at theright location.

No additional course packages will be added to the ones already sent after the college has verified and approved the packaging file.

Local Agreement

The Key College Contact and any other college personnel, as required, and the student government(s) are to develop, review, and/or revise a college process to administer the survey based on the protocol set out in this document. The student government is to play a significant role in the administration of the survey and is to certify that the survey has been conducted according to the local agreement that formalizes the internal survey administration. College and student government(s) and/or members of the student body are to:

develop a local agreement that formalizes the internal survey administration;






Operating Procedure


participate in the design and/or modifications of the survey administrationprocess;

support the administration process;

observe the overall college survey process;

certify3 that the survey was conducted according to the protocol by signing thecertification section on each survey package envelope (see Appendix G); and

release publicly, particularly to stakeholders who provided time and input to thesurvey/data collection (i.e., students and college staff), the survey results as perthe framework for Release of Key Performance Indicator Data.

3 To comply with the Ministry’s audit requirements, the independent Supplier is required to obtain certification from the survey administrators (student and/or college staff) that the survey was conducted according to the protocol.

A local agreement is to include:

a general statement of agreement between the college and studentgovernment(s);

a statement of the responsibilities and roles, in general or specific terms, of thecollege and the student government(s);

the promotion and communication of the survey and process to students andcollege employees by such means as flyers, memos, e-mails, class and meetingpresentations, college newspapers, etc.;

the time period or term for the signed agreement;

the recruitment and training of students and staff for survey administration;

the storage and security of surveys during the survey process;

a process for debriefing or review of the survey administration process to plan forthe next survey cycle;






Operating Procedure


the discussion and sharing of the results and participation in the development ofstrategies to address raised issues; and

a format for the release of the survey results to key stakeholders: the studentsand college staff.

An agreement is to be drawn up for each of the surveys or may cover all three surveys. The local agreement(s) must be submitted to the Ministry in the month of:

May if the advanced June survey is to be conducted;

October if the advanced November survey is to be conducted; and

January if only the formal survey is to be administered.

The specific due dates for the submission of this agreement are listed in Appendix A.

The agreement is to be signed by an executive member of the college and the president(s) of the student government/association. A copy of the signed agreement is to be e-mailed to the Postsecondary Accountability Branch Director at [email protected] and the original is to be mailed to the Director, Postsecondary Accountability Branch, Ministry of Training, Colleges and Universities, Mowat Block, 7th Floor, 900 Bay Street, Toronto, ON, M7A 1L2.

If a college and student government(s) cannot reach an agreement, the college is still expected to ensure that the survey is conducted using college staff in accordance with the survey administration procedures in this document. A letter is to be e-mailed to the Postsecondary Accountability Branch director at [email protected] outlining the reasons why an agreement could not be reached.

Student Profile Data File

The Key College Contact is to send to the Supplier, via the online system, the student profile data (spd) (see Appendix E) in January, by the date specified in Appendix A. All students enrolled with the college in the Winter semester, whether on campus or not, are to be included in this file. As well, all students that were available for the advanced surveys in June through the Non-Traditional Pattern of Program Delivery option; and all field, co-op, and clinical placement students well as those approved through the Non-Traditional Pattern of Program Delivery option that completed the advanced survey in








Operating Procedure


November are to be included in this student file. An effective date of January 31 of the year in which the formal survey is conducted is to be used to determine a student's age.

The online system checks the file format automatically and displays any errors in the file. If the file is correct, the Key College Contact is advised and given the opportunity to send the file to the Supplier. If the file contains errors, the Key College Contact is to ensure that the appropriate revisions are made to the file and then resubmit the file in the same manner. Some errors will be flagged as critical and must be corrected before the file will upload successfully. Other errors will result in a warning message only, that will not prohibit the file from being uploaded, but will allow the Key College Contact to make a change if deemed necessary.

Communication

Informing students about the Student Satisfaction and Engagement Survey through posters or newspapers is to be done in an impartial manner to avoid compromising the validity and reliability of the data. Students are to be informed that:

All survey responses are valid.

Students should select a response that best reflects how they feel.

The survey results are not currently used to distribute performance funding tocolleges.

Survey Protocol

The college is responsible for the overall survey process and for ensuring that the Student Satisfaction and Engagement Survey is conducted in a controlled, impartial environment that will ensure the integrity, consistency, and confidentiality of the data collected. The colleges and the Supplier are required to carry out the same survey administration and distribution procedures for both advanced surveys and the formal survey.


establish internal college mechanisms for the effective and efficient distributionand return of survey packages;

convey the information about the internal college distribution procedures to theSupplier in accordance with the deadlines listed in Appendix A; and






Operating Procedure


work with the student government(s) to administer the survey according to thelocal agreement and in keeping with this operating procedure.

Student Survey Population

The Student Satisfaction and Engagement Survey is to be conducted with all students enrolled in postsecondary programs of instruction approved by the Ministry for funding through the general purpose operating grant or enrolled in CODA programs of instruction approved for funding through CODA contracts, including those enrolled in applied degree programs, except for:

Students who are:

registered in their first semester of their program of instruction; or

have been given advanced standing and are in the program of instruction forthe first time.

Students who are registered as part-time and are NOT pursuing a certificate,diploma or

Students enrolled in the collaborative nursing programs (MTCU 81400) are to be included in the survey, as long as the students are in at least their second semester in a college setting.

Students with Disabilities

Colleges, as direct providers of postsecondary educational programs and services, have the legal responsibility under the Ontario Human Rights Code and the Canadian Charter of Rights and Freedoms to accommodate persons with disabilities.

Students with disabilities, who require assistance and/or alternative means to complete the survey, are to be accommodated as determined by each college. Colleges are to work with their Offices for Students with Disabilities to ensure that appropriate accommodations and/or alternative survey formats, as required, are arranged prior to the survey period. This is to enable all students to complete their surveys during the two-week survey period. In the event additional time is required, students with disabilities may be accommodated in advance of the survey period by up to one week.

Colleges may request from the Supplier the following alternate survey delivery options:

applied degree






Operating Procedure


Enlarged survey form (11” by 17”)

Braille survey

Telephone survey

On-line – non-visual (audio)

On line - visual

Students with disabilities may complete the survey at the college's Office for Students with Disabilities. Students with disabilities may have the assistance of an aide or assistant to complete the survey and may also use assistive devices to complete the survey. Colleges are to ensure that any accommodations for students with disabilities have been addressed prior to completing and forwarding the packaging file data to the Supplier.

Colleges may request the Supplier, through the completion of the packaging file, to prepare a survey package for students with disabilities by placing "Students with Disabilities” in the course name field section of the packaging file.

If students with disabilities complete an Enlarged or Braille paper-based survey (individually or in small groups) at the Office for Students with Disabilities, the following steps are to be followed:

1) The completed surveys MUST be sealed in a college envelope to ensure theintegrity of the process.

2) Individually sealed college envelopes are to be inserted into the Supplier'senvelope on which the certification section is printed.

3) The Supplier's envelope is to be sealed and the certification section signed.

4) The Supplier's envelope is to be sent to the Supplier by the last due date, asspecified in Appendix A.

Distance Education Students

In keeping with the protocol intent, students who are studying at home, using the correspondence method, are NOT to be included in this survey. Only those distance






Operating Procedure


education students who are attending a class at a distant site and for whom a site administrator and/or student monitor is in attendance are to be surveyed.

When colleges are preparing the packaging file, they are requested to provide the Supplier with the name of the distance education site(s) by the earliest date possible in October for the advanced survey and in January for the formal survey, as listed in Appendix A.

Advanced Standing

Students with advanced standing who have reached the second semester of their program or beyond should participate in the survey.

Students who have been given advanced standing and are actually attending the program for the first time should NOT be asked to participate in the survey. These students have not been in the program long enough to effectively evaluate the program, services, and learning experiences.

Part-time Students

Part-time students pursuing a certificate or diploma in a Ministry funded full-time postsecondary program of instruction should participate in the survey. The mode or time of the course delivery should not be a factor.

If a part-time student is NOT pursuing a certificate, diploma or an applied degree in a Ministry funded full-time postsecondary program of instruction, they should NOT be asked to participate in the survey.

Non-Traditional Pattern of Program Delivery

Colleges that provide program delivery in a non-traditional pattern, which in some cases under the existing guidelines is excluded from the Student Satisfaction and Engagement Survey process, can have these programs considered for inclusion in the survey process and the overall Student Satisfaction KPI (see Appendix L for examples).

If all other requirements of the protocol for Student Satisfaction and Engagement Survey administration have been met, colleges may write to the Postsecondary Accountability Branch director at colleges.branch@ Ontario.ca requesting that a program be included in the survey process. The request must include details of the program delivery and be signed by the college president.







Operating Procedure


For the June advanced survey, colleges are to forward their requests to the Ministry in early April by the date specified in Appendix A.

The Ministry will review each request and advise the college and the Supplier of its decision in advance of the mid-April date, on which colleges are to submit to the Supplier a list of programs by campus, and the total number of surveys required for the advanced June survey.

For the November advanced survey colleges are to forward their requests to the Ministry at least one month prior to the mid-September date on which colleges are to submit to the Supplier a list of programs by campus, and the total number of surveys required for the advanced survey and the formal survey.

The Ministry will review each request and advise the college of its decision in advance of the mid-September date.

If approval is given, the results will be included in the college's Student Satisfaction KPI for the survey cycle year when the surveying occurs.

Survey Administration Procedures

Survey package envelopes have instructions for survey administrators (see Appendix F) affixed to the envelope and contain a standard instruction sheet for each student being surveyed. These instructions provide important information to survey administrators and to students. In explaining the survey to students, survey administrators are to be careful not to inadvertently bias the survey results by redefining the response scale or encouraging students to select one response over another. Survey administrators are to explain to students that:

the five-point scale is frequently used in satisfaction surveys;

all responses on the scale are valid;

students should choose a response that best describes how they feel; and

the survey results are not currently used to fund colleges;

Survey administrators are to avoid making statements like the following, which may lead to bias in the results:

"Give us good ratings and we will get money to improve programs."






Operating Procedure


"Give us poor ratings and we will not get money to improve programs."

"Only 'Very Satisfied' and 'Satisfied' count as positives for the KPI."

"We did poorly last time, but we have made improvements and we are hopingthis will show in the results."

"We were just behind college ABC last year; this year we hope to pass it in theratings."

"’Neither Satisfied nor Dissatisfied’ is:

a negative response."

is the same as being ‘Dissatisfied’ in the KPI calculation."

is not a useful response."

It takes approximately 10 minutes to review the instructions with the class before students begin to complete the survey. Completion of the surveys takes approximately 25 minutes.

Students who are registered in a specific program, but who may be attending a class of another program when the Student Satisfaction and Engagement Surveys are distributed in class, are to be asked to declare if they have already completed a Student Satisfaction and Engagement Survey. Students who should NOT complete the survey include the following:

Students who completed this survey in the past 10 days or the advanced survey in June or November;

Students who are:

registered in their first semester of their program; or

have been given advanced standing and are in the program for the first time.

Students who are registered as part-time and are NOT pursuing a certificate, diploma or applied degree.






Operating Procedure


The survey administrators are required to do the following:

Administer, with the active participation of the student government(s), thesurveys as per the local agreement and the Instructions for SurveyAdministrators attached as Appendix F. In the event that a student does notparticipate in the administration of the survey, the college staff member is to doso.

Ensure that the survey packages and pencils are available for distribution.

Distribute the survey (see Appendix H) and survey instruction sheet to all eligiblestudents in classes that have been identified during the "survey week". A studentrepresentative from each class may be asked to participate in the administrationprocess. Faculty, learning facilitators, or survey administrators are to stay in classduring the completion of the survey in case questions or difficulties arise but areasked to allow the students to complete the surveys in confidence.

Conduct the survey during the beginning 20 to 30 minutes of the class.

Ensure that the certification section, which is printed on the envelope andconfirms that the survey process was conducted according to the instructions, issigned (see Appendix G).4

4 This certifies that, to the best knowledge of the student and/or staff member, the survey was conducted according to the standard protocol. The Supplier requires certification to satisfy the Ministry’s verification requirements.

Note: The surveys will NOT be processed for the colleges but will be recorded if the following conditions occur:

Envelopes are unsealed or incorrectly sealed

No person’s name appears in the certification section

No certifying signature appears in the certification section

Surveys are returned loose

Surveys are returned in non-standard packages.






Operating Procedure


Other surveys will be recorded and processed and flagged as problem surveys if the following occur:

Packages with more completed surveys than the number of students in the class

Packages with no numbers

Packages with incomplete numbers, for example, packages for which the number of students in the class is missing

Surveys with incorrect or no program codes

Return of Completed Surveys

For both the advanced survey and formal survey, each college is to:

identify a location(s), agreed upon by the college and student government(s) through the local agreement, for the return of the completed surveys contained in sealed envelopes. The college location is to be central, secure, and lockable;

apply a courier package tracking label to each box or PuroPak sent to the Supplier. The Supplier will provide 10 of these tracking labels for each advanced (June & November) survey and 30 for the formal (Winter) survey. The tracking labels are to be used in consecutive order. The Supplier will expect to receive packages with tracking labels in consecutive order. This will enable the Supplier to inform colleges if any packages have not arrived and to recommend that missing packages be traced immediately; and

return the completed survey packages to the Supplier by next-day courier service by the dates specified in Appendix A. All unused surveys are also to be included in survey packages and returned to the Supplier.

Distribution of Survey Results and Reports

The data collected by the advanced and formal Student Satisfaction and Engagement Surveys are aggregated and used for administrative and statistical purposes of the colleges and the Ministry. Only aggregate data are reported, and all individual responses are confidential.






Operating Procedure


In light of the Ontario Freedom of Information and Protection of Privacy Act, confidentiality is maintained by not publishing information for categories in which the total number of individuals is less than five.


Notify the Supplier via e-mail that the comparative report colleges receive is to be provided in pdf format rather than hard copy by the due date specified in Appendix A. An FTP site will be provided to obtain reports and data if PDF format is selected.

Distribute to the appropriate staff in the colleges the following data and reports received from the Supplier for the surveys:

the raw survey results in either an ASCII file or dbf format; and

tabulated reports as outlined in Appendix I.

Release of Key Performance Indicator Data

The colleges’ KPI results are announced annually in the spring. Individual colleges may release their results to the public after the announcement. In publishing the KPI results, colleges and the Ministry are to adhere to the following requirements:

Colleges are not to be ranked by KPIs; rather, show areas of strength and that performance is at acceptable levels.

All KPIs are to be reported together as no one indicator provides a complete picture of college performance.

The context and unique local circumstances are to be provided as background for the Student Satisfaction KPI results.

KPI data are to be shared publicly, particularly with stakeholders who provided time and input to the surveys or data collection (i.e., students, graduates, employers).

KPI data are to be reported as quickly as possible depending on data readiness.

http://www.ontario.ca/laws/statute/90f31






Operating Procedure


Review and Adjustment of the Student Satisfaction KPI

The Ministry undertakes a review of the KPI data collected during the previous collection cycle and makes adjustments to ensure that only data obtained from eligible students are used in the calculation of the KPI (see Appendix J). A memo detailing the review and adjustment process is sent to the college president, registrar, and Key College Contacts for the Student Satisfaction and Engagement Survey.

The Key College Contact for the Student Satisfaction and Engagement Survey is to ensure that:

The appropriate college staff review the KPI summary report for the Student Satisfaction and Engagement Survey provided by the Supplier.

Any requests from the college to adjust the surveys included in the KPI calculation are sent to the Ministry for consideration.

The student satisfaction certification form is signed by the college president and returned by the due date.

Further Information

For further information regarding this operating procedure, click on the Contact link provided in the Contacts section of the web site to consult with the appropriate Ministry contact. The Supplier administering the Student Satisfaction and Engagement Survey can be reached via the contact information found in Appendix B.

Summary of Responsibilities

Colleges of Applied Arts and Technology

The college is responsible for:

appointing a Key College Contact to work with the Supplier and the Ministry on the administration of the KPI Student Satisfaction and Engagement Survey;

providing all necessary information to the Supplier as specified in this operating procedure;

working with the student government(s) to jointly develop a local agreement that formalizes the internal survey process;







Operating Procedure


conducting the Student Satisfaction and Engagement Survey in a controlled, impartial environment;

keeping college staff and students informed about the KPI process and results; and

working with the Ministry, through the joint College-Ministry KPI Policy Committee and the KPI Operations Subcommittee, to address issues and make recommendations concerning the administration of the Student Satisfaction and Engagement Survey.


The Ministry is responsible for:

administering the KPI initiative:

providing the KPI results for the annual announcement;

working with the colleges, through the KPI Policy Committee and the KPI Operations Subcommittee, to explore issues concerning the administration of the Student Satisfaction and Engagement Survey and make decisions on changes that may be required;

revising operating procedures as required, taking into account the recommendations of the KPI Policy Committee and the KPI Operations Subcommittee and the impact of the changes on the colleges; and

using data to advise and inform the government about the college system’s achievement of its mandate and in planning and policy-making for the college system.






Operating Procedure


Appendix A: Key Dates

ADVANCED JUNE SURVEY, June 8 to June 19, 2015 (optional) (Colleges may choose to participate in the Advanced June Survey to survey those students in a non-traditional pattern of program delivery for which Ministry approval to survey has been obtained).

DATE ACTIVITY

April 24, 2015 Colleges declare to the Ministry their decision to survey in June 2015 and submit to Ministry a request to review for inclusion in the survey process program(s) delivered in a non-traditional pattern.

May 1, 2015 Colleges send up to five college-specific questions to the Supplier via a password protected web-based system. As this is the start of a new survey process, colleges must re-submit questions, even if the questions have not changed from the previous year. However, previous questions will already be pre-loaded into the web-based system for sending so that they do not need to be re-typed.

May 1, 2015 Ministry sends Supplier a list of programs approved for the June advanced survey.

May 1, 2015 Supplier develops and revises the final survey forms, survey package envelopes and student instructions.

May 4, 2015 Colleges provide the service provider, via a password protected web-based system, with an electronic list of college programs for the June survey by campus for the student instruction sheet. Colleges provide the service provider with the total number of surveys required for the June advanced survey, the advanced survey in November and the formal survey in February.

May 4, 2015 Supplier sends the college-specific questions on the survey form to the colleges by e-mail for approval.

May 8, 2015 Colleges provide the Ministry with the local survey administration agreements and/or revisions between colleges and student government(s).

May 11, 2015 Colleges approve the college-specific questions on the survey form by replying in e-mail.

May 20, 2015 Supplier sends the college survey instruction sheet to the colleges by e-mail for approval.






Operating Procedure


DATE ACTIVITY

May 22, 2015 Colleges upload, via a password protected web-based system, the electronic packaging file for administering the surveys. Colleges complete the packaging verification process which includes sending the electronic packaging file, informing the Supplier of the designated drop-off location(s) to which a courier is to deliver the survey package envelopes and indicating how they would like the Supplier to sort the survey packages.

May 22, 2015 Colleges approve their survey instruction sheet by replying in email.

May 25-29, 2015 Supplier prepares the survey packages.

May 29-June 5, 2015 Supplier sends the survey packages and a corresponding number of student instruction sheets to colleges by courier.

June 8-19, 2015 Colleges conduct the advanced survey.

June 11, 2015 Colleges courier the survey packages completed on June 8 to June 10 to the Supplier by next-day courier service.

June 17, 2015 Colleges courier the survey packages completed on June 11 to 16 to the Supplier by next -day courier service.

June 24, 2015 Colleges courier ALL remaining survey packages to the Supplier by next-day courier service.

ADVANCED NOVEMBER SURVEY, November 2 to November 13, 2015 (optional) (Colleges may choose to participate in the Advanced November Survey to survey ONLY (1) their field, co-op, and clinical placement students who will not be available during the February 1 to February 12, 2016 survey period, and (2) those students in a non-traditional pattern of program delivery for which Ministry approval to survey has been obtained). (See Appendix B for the Supplier's address.)

DATE ACTIVITY

August 21, 2015 Colleges declare to the Ministry their decision to survey in November 2015 and submit to the Ministry a request to review for inclusion in the survey process program(s) delivered in a non-traditional pattern.






Operating Procedure


DATE ACTIVITY

September 11, 2015 Colleges that did not participate in the advanced June survey send up to five college-specific questions to the Supplier via a password protected web-based system. As this is the start of a new survey process, colleges must re-submit questions, even if the questions have not changed from the previous year. However, previous questions will already be pre-loaded into the web-based system for sending so that they do not need to be re-typed.

September 11, 2015 Ministry sends Supplier a list of programs approved for the November advanced survey.

September 18, 2015 Supplier develops and revises the final survey forms, survey package envelopes and student instructions.

September 25, 2015 Colleges that did not participate in the June survey provide the Supplier, via a password protected web-based system, with an electronic list of college programs for the November survey by campus for the student instruction sheet. Colleges provide the Supplier with the total number of surveys required for the advanced survey in November and the formal survey in February.

Colleges that did participate in the June survey may provide additional programs for the instruction sheet through the same process utilized in the June. The file submitted will be used to append more programs to the current list.

October 2, 2015 Colleges provide the Ministry with the local survey administration agreements and/or revisions between colleges and student government(s).

October 2, 2015 Supplier sends the college-specific questions on the survey form to the colleges by e-mail for approval.

October 9, 2015 Colleges approve the college-specific questions on the survey form by replying in email.

October 13, 2015 Supplier sends the college survey instruction sheet to the colleges by e-mail for approval.

October 16, 2015 Colleges upload, via a password protected web-based system, the electronic packaging file for administering the surveys. Colleges complete the packaging verification process which includes sending the electronic packaging file, informing the Supplier of the designated drop-off location(s) to which a courier is to deliver the survey package envelopes and indicating how they would like the Supplier to sort the survey packages.

October 16, 2015 Colleges approve their survey instruction sheet by replying in email.






Operating Procedure


DATE ACTIVITY

October 19–23, 2015 Supplier prepares the survey packages.

October 23–28, 2015 Supplier sends the survey packages and a corresponding number of student instruction sheets to colleges by courier.

November 2-13, 2015 Colleges conduct the advanced survey.

November 5, 2015 Colleges courier the survey packages completed on November 2 to 4 the Supplier by next-day courier service.

November 10, 2015 Colleges courier the survey packages completed on November 5 to 9 to the Supplier by next -day courier service.

November 18, 2015 Colleges courier ALL remaining survey packages to the Supplier by next-day courier service.






Operating Procedure


FORMAL SURVEY, February 1 to February 12, 2016

DATE ACTIVITY

November 13, 2015 Colleges that did not participate in the advanced June survey or the advanced November survey send up to five college-specific questions to the Supplier via a password protected web-based system. As this is the start of a new survey process, colleges must re-submit questions, even if the questions have not changed from the previous year. However, previous questions will already be pre-loaded into the web-based system for sending so that they do not need to be re-typed.

November 13, 2015 Ministry sends Supplier a list of programs approved for the formal survey.

November 21, 2015 Supplier develops and revises the final survey forms, survey package envelopes and student instructions.

November 27, 2015 Colleges that did not participate in the June survey or the November survey provide the Supplier, via a password protected web-based system, with an electronic list of college programs for the formal survey in February by campus for the student instruction sheet. Colleges provide the Supplier with the total number of surveys required for the formal survey in February.

Colleges that did participate in the June survey or the November survey may provide additional programs for the instruction sheet through the same process utilized in the June and/or the November survey. The file submitted will be used to append more programs to the current list.

November 27, 2015 Colleges provide the Ministry with the local survey administration agreements and/or revisions between colleges and student government(s).

November 27, 2015 Supplier sends the college-specific questions on the survey form to the colleges by e-mail for approval.

December 2, 2015 Colleges approve the college-specific questions on the survey form by replying in email.

December 7, 2015 Supplier sends the college survey instruction sheet to the colleges by e-mail for approval






Operating Procedure


DATE ACTIVITY

January 15, 2016 Colleges upload, via a password protected web-based system, the electronic packaging file for administering the surveys. Colleges complete the packaging verification process which includes sending the electronic packaging file, informing the Supplier of the designated drop-off location(s) to which a courier is to deliver the survey package envelopes and indicating how they would like the Supplier to sort the survey packages.

January 18–22, 2016 Supplier prepares the survey packages.

January 19–28, 2016 Supplier sends the survey packages and a corresponding number of student instruction sheets to colleges by courier.

January 29, 2016 Colleges send the Supplier, via the online system, the student profile data file and include students from the advanced June and November surveys as well as the formal survey.

January 29, 2016 Colleges advise Supplier of their desired format for the comparative report via the online system.

January 27-29, 2016 Colleges conduct the formal survey – voluntary three-day option.

February 1, 2016 Colleges courier the survey packages completed on January 27 to 29 to the Supplier by next-day courier service.

February 1-12, 2016 Colleges conduct the formal survey.

February 4, 2016 Colleges courier the survey packages completed on February 1 to3 to the Supplier by next-day courier service.

February 8, 2016 Colleges courier the survey packages completed on February 4 to5 to the Supplier by next-day courier service.

February 11, 2016 Colleges courier the survey packages completed on February 8 and 10 to the Supplier by next –day courier service.

February 15, 2016 Colleges courier the survey packages completed on February 11 and 12 to the Supplier by next-day courier service.

February 17, 2016 Colleges courier ALL remaining postsecondary survey packages to the Supplier by next -day courier service.

February 24, 2016 Colleges courier ALL remaining apprenticeship survey packages to the Supplier by next -day courier service.






Operating Procedure


DATE ACTIVITY

March 2016 Ministry and the colleges receive electronic and hard copies of the Student Satisfaction KPI data and each college receives its own raw data

November– December 2016

The final review and adjustment process is completed.

January 31, 2017 Supplier provides all colleges and the Ministry with a final KPI report showing each college’s adjustments.






Operating Procedure


Appendix B: Supplier's Contact Information

For more information on the Student Satisfaction and Engagement Survey process, contact:

CCI Research Inc. 71 Broadway Orangeville, ON L9W 1K1

Tel: 1-888-246-8068 Fax: 1-888-938-8554

E-mail: [email protected]

KPI password protected web-based system:

https://www.cci-survey.ca/kpi/ontario

Service Provider’s Web site: https://www.cci-research.com

https://www.cci-research.com

https://www.cci-survey.ca/kpi/ontario







Operating Procedure


Appendix C: Student Satisfaction and Engagement Survey Instructions for Students

KPI STUDENT SATISFACTION AND ENGAGEMENT SURVEY

Ministry of Training, Colleges and Universities and XXXXXXX College

This is a great opportunity for you to give meaningful feedback to the college that will make a difference!

QUICK START STUDENT INSTRUCTIONS

Make heavy, dark marks. PLEASE USE A PENCIL.

Do not use check marks to select boxes.

If you erase, please erase completely.

Please do not make any other marks on this form.

1. You have already completed the survey in the past ten (10) days.2. You are (a) registered in the first semester of your program, or (b) have been

given advanced standing and are in the program for the first time.

If you have any concerns or suggestions regarding the survey administration

process, please provide feedback at www.kpifeedback.ca (you may keep this form).

It is important to identify your program by placing the three-digit alpha-numeric program code on the front of the survey. See the back of this sheet for the program names and the three-digit alpha-numeric program codes. If you are uncertain about the program name and/or code, please ask the survey administrator in the class for assistance.

Please DO NOT complete the survey, (simply return the blank survey to the

survey administrator) if:

http://www.kpifeedback.ca/studentsatisfaction/






Operating Procedure


---------------------------------------------

3. You are registered as a part-time student and are NOT pursuing a certificate,diploma or

Additional Information

Background of the Survey This is a great opportunity for you to give meaningful feedback to the college that will make a difference. This feedback will help contribute to the development of strategies to improve college programs and resources, improve the quality of college life and provide greater assurances that graduates will be “work ready”. Six months after you graduate, you will again be provided an opportunity to give feedback when you are surveyed by an independent service provider for the Graduate Employment and Graduate Satisfaction survey.

Purpose of the Survey The Ministry of Training, Colleges and Universities believes that colleges have the responsibility to provide you with high quality and relevant programs that meet your learning needs. This survey measures how satisfied you are with the quality of your learning experience, support services and educational resources and will provide an indication of how well these needs are being met. The survey also looks in more detail at your student experiences, including the demands on your time. The results of the survey will also provide information about areas of strength or concern so that institutions can develop strategies for improvement. Anonymous information from the survey is provided to the Ministry to produce the annual aggregated employment profile of graduates and to report on the performance of colleges. The survey results are not currently used to distribute performance funding to colleges.

Confidentiality The information gathered from this survey is confidential.

Student names are not on the survey and are not recorded anywhere;

Surveys are not numbered; and

Faculty/learning facilitators are not identified on the survey.

Please be assured that when you complete this survey, information reported will not identify you in any way. Neither the Colleges nor the Ministry will see the survey results by class. Programs of less than five (5) students will not be identified to the Colleges.

Length of Survey It should take 20 – 30 minutes to complete the survey. Please take your time and read the instructions and the survey questions carefully.

applied degree.

------------------------------------------






Operating Procedure


Appendix D: Packaging File

A separate packaging file* should be completed for each separate course or program in which students will be surveyed and forwarded to the Supplier, via a password protected web-based data file uploader, no later than May 22, 2015 for those colleges wishing to undertake the advanced June survey, October 16, 2015 for those colleges wishing to undertake the advanced November survey and no later than January 15, 2016 for the formal survey. (See Appendix B for the Supplier's address.)

*Packaging file are fields of college-specific information required by the Supplier to package the surveys by course, program, courseor program section, college, and location. This information will be placed on the outside of the envelopes containing the surveys thatwill be sent to each college.

The college code to be used in the Packaging Record Layout is defined in Appendix K.

Packaging Record Layout Fields Length (Characters) Position

1. College code (e.g., HUMB) 4 1–4

2. Campus or location (e.g., Newnham) 30 5–34

3. Course or program name (e.g., Economics) 45 35–79

4. Course or program code (i.e., short form or abbreviationof name code, EC100)

12 80–91

5. Course or program section number (e.g., A) 5 92–96

6. Number of surveys required in the packages 3 97–99

7. Drop-off location 60 100–159

8. Campus location postal code (6 characters) 6 160-165

Optional fields have been provided to assist colleges and to facilitate in the distribution process. Optional Fields

9. Course section contact name 30 166–195

10. Day of the week (1-Monday, 2-Tuesday, 3-Wednesday,4-Thursday, 5-Friday, 6-Saturday, and 7-Sunday.)

9 196–204

11. Date (yy/mm/dd) 8 205–212

12. Room number 8 213–220

13. Time 8 221–228






Operating Procedure


Appendix E: Student Profile Data File by Program

As part of the administration for the Student Satisfaction and Engagement Survey process, there is a requirement to gather data on the student profile for each program of study that will be surveyed in the June and November advanced surveys and the formal survey. All students enrolled with the college in the Winter semester, whether on campus or not, are to be included in this file. As well, all students that were available for the advanced surveys in June through the Non-Traditional Pattern of Program Delivery option; and all field, co-op, and clinical placement students well as those approved through the Non-Traditional Pattern of Program Delivery option that completed the advanced survey in November are to be included in this student file. A precise audit of the KPI responses cannot be done since the students will not be identified in the survey process. In lieu of an audit, the student profile data is compared to the survey respondents profile to ensure that a representative body of students was surveyed. A separate student profile data file is required only once but should include student profile data for both the advanced and formal surveys. This file should be forwarded to the Supplier, via a password protected web-based data file uploader, no later than January 29, 2016. (See Appendix B for the Supplier’s address.)

The college code to be used in the Student Profile Data Record Layout is defined in Appendix K.

An effective date of January 31, 2016 will be used to determine a student's age.

Student Profile Data Record Layout (Postsecondary) Fields Length (characters) Position


2. -DIGIT CODE assigned by the Supplier or the college 3 5–7

3. College program name (e.g., Law & Security) 45 8–52

4. College program name (long version for some reports) 75 53-127


12 128-139

6. MTCU code 5 140-144

7. Approved program sequence (APS) number 5 145-149

8. Campus or location (e.g., Newnham) 30 150-179

9. Campus code (i.e., short form or abbreviation of campus orlocation, e.g., NM)

8 180-187

10. Program TOTAL enrolment number (NOT in their first semester) 3 188-190

THREE






Operating Procedure


enrolled in full-time, postsecondary program approved for Ministry funding through CFF

11. Program full-time enrolment number (NOT in their firstsemester) enrolled in full-time, postsecondary programapproved for Ministry funding through CFF

3 191-193

12. Program part-time enrolment number (NOT in their firstsemester) enrolled full-time, postsecondary program approvedfor Ministry funding through CFF

3 194-196

13. Number of males in target group 3 197-199

14. Number of females in target group 3 200-202

15. Number with English as a first language in target group 3 203-205

16. Number with French as a first language in target group 3 206-208

17. Number with Other as a first language in target group 3 209-211

18. Number ages less than 21 in target group 3 212-214

19. Number ages 21–25 in target group 3 215-217



22. Number ages more than 35 in target group 3 224-226


Field Notes

10 to 22. Complete a student profile data record for each full-time, postsecondary program approved for Ministry funding through the College Funding Framework (CFF), . Enter the number of students (NOT in their first semester) enrolled in the Winter semester (January term). This should include students in the June and November survey who are not available for surveying in February.






Operating Procedure


Appendix F: Instructions for Survey Administrators

KPI STUDENT SATISFACTION SURVEY PACKAGE ENVELOPE

Student Satisfaction Survey Pkg# 123456

New Liskeard OP2000 3AO Office Administration Mary Smith COLL WED L567

(College drop-off location inserted for completed survey packages)

Please administer this survey at the beginning of class.

Please do not hand out surveys until you have completed steps 1 to 4.

Step 1 Confirm that the information on this survey package envelope is correct (see label above):

(1) college name, (2) campus, (3) course name, (4) course code, (5)course section number

Confirm that pencils are available.

Step 2 Inform the students about the survey:

This survey is an opportunity for students to give the college meaningful feedback that will make a difference.

This feedback will contribute to the development of strategies to improve college programs, resources and your overall experience.

This feedback will improve the quality of college life and provide greater assurances that graduates will be "work ready."

The five-point scale is frequently used in satisfaction surveys.

All responses on the scale are valid.






Operating Procedure


Students should choose a response that best describes how they feel.

The survey results are not currently used to fund colleges.

Ask students enrolled in the collaborative nursing program (MTCU 81400) to focus on their college experience when completing the survey.

Step 3 Determine how many students are in each of the following categories and place the respective number in each of the four boxes below (if zero, place “0” in the box):

Total number of students in the classroom at the time of administrating the survey:

Students who should NOT complete the survey include:

1. Number of students who have completed this survey inthe past 10 days or during the weeks of June 8 to19,2015 or the weeks of November 2 to 13, 2015.

2. Number of students who are (a) registered in their firstsemester of their program or (b) have been givenadvanced standing and are in the program for the firsttime.

3. Number of students registered as part-time and NOTpursuing a certificate, diploma .

Step 4

1. confirming the names of the programs in which students areregistered (print the names on the board or flip chart);

2. instructing the students to fill in the appropriate code for theprogram that is identified on the chart provided on the back of the

or applied degree

Instruct the students to use a pencil to make a solid, thick stroke through the selection boxes. Other marks on the survey are not permitted. An erasure can be made, but it must be done completely.

Instruct the students in completing Section A of the survey by:






Operating Procedure


instruction sheet (print the relevant program codes on the board or flip chart);

3. asking if anyone is currently taking courses in more than oneprogram. If so, ask the student to fill in the program code for theprogram in which he or she is currently registered.

Instruct the students to read the student instruction sheet prior to completing the survey. If students are having problems with program names and/or codes, please encourage them to ask the survey administrator for assistance.

Step 5 Hand out the instruction sheets and the survey to eligible students. DO NOT hand out the survey to students who are ineligible.

Faculty, learning facilitators, and survey administrators are encouraged to stay in case questions or difficulties arise during the completion of the survey, but are asked to allow the students to complete the surveys in confidence.

Step 6 Collect the completed surveys and place them and any blank surveys in the original envelope. DO NOT INCLUDE PENCILS.

Step 7 Seal the envelope (the envelope lip will seal on contact with the envelope). Sign the certification section on the envelope confirming that the process outlined in these instructions has been followed.

Step 8 Deliver the envelope of completed surveys to the designated drop-off location(s) as indicated at the top of this survey package envelope.






Operating Procedure


Step 9 Ensure that a number is in every box in Step 3, that you have printed and signed your name in Step 7, and that the package is sealed.

Appendix G: Process Verification

To ensure that the KPI data are accurately compiled and reported, the Supplier is required to have an independent public accountant examine the KPI data tabulation and reporting processes according to the Ministry’s Guidelines to Examine Collection and/or Reporting Procedure.

According to the guidelines, the Supplier is to reconcile the data received from the colleges with the certification signed by a college official and/or a student from the student body, which certifies that to the best of their knowledge and belief the survey was conducted according to the protocol.

The certification located on the survey package envelope (see below) must be completed by the survey administrator. Survey envelope packages on which this section is incomplete will be rejected.

CERTIFICATION SECTIONPlease ensure you have:(1) filled in the numbers • STEP 3,(2) sealed the envelope, and(3) printed and signed your name, entered the date

in this box indicating that the survey has beenconducted according to the protocol.

Print First and Last Name Here

Full Signature

Today's Date:

To comply with the verification requirements of the Ministry, the colleges' survey administrators must completethis Certification Section to verify that the survey was conducted according to the instructions.






Operating Procedure


Appendix H: Student Satisfaction and Engagement Survey






Operating Procedure







Operating Procedure







Operating Procedure







Operating Procedure







Operating Procedure


Appendix I: Survey Results and Reports

Each college will receive its own results from the formal survey or the advanced and formal surveys combined, including responses to its specific questions. The Key College Contact receives:

the raw survey results in either ASCII file or dbf format; and

tabulated reports detailed below.

A comparative report will be provided to each college that includes a provincial average, the college average, and each program average, in hard copy or pdf format, and tab-delimited electronic format. Colleges will receive the above reports in March as shown in Appendix A.

In addition to the above, each college will receive in ASCII format (suitable for WordPerfect or Microsoft) a report that includes the following:

Administrative Summary:

This section documents the number of processed survey packages and thenumber of rejections; and provides a list of the rejected packages bycourse/program name with the reasons for rejection.

Survey Results:

Colleges receive a report that will include the total number in the class, the totalnumber completing a survey, and the return rate (percentage of students in theclass who completed a survey).

The Student Satisfaction KPI for each college is defined as the averagepercentage of students who completed the survey and who responded that theywere satisfied or very satisfied with the quality of their learning experience,support services, and educational resources.

The Student Satisfaction KPI calculation is derived from responses of allcapstone questions combined. The KPI calculation is based on students who:

answered Question 2 (In which semester/term are you currently enrolled?);






Operating Procedure


* surveys not from the relevant student population, i.e., students in first semesterwho completed the survey

** assumes these numbers are mutually exclusive

indicated that they are in semester 2 or above;

answered ALL capstone questions.

Each college will receive for each program and all programs combined the percentage of respondents who answered in the categories of "Satisfied"/"Very Satisfied", "Neither Satisfied nor Dissatisfied", and "Dissatisfied"/"Very Dissatisfied" to each of the capstone questions and all combined.

In addition, colleges will receive the distribution of responses (numbers and percentages) for all survey questions and the total number of students registered at the college.

Return Rate Calculation:

The return rate is calculated using numbers from the certification section on thesurvey package envelope. If the numbers are incomplete or contradictory, theywill be excluded.

The return rate is:

Total # of surveys in class – # of first semester students (Q2 ' 1 – adjustment*) Total # of students in class – (# of students who already completed the survey + # of students in first semester including students with advanced standing + # of part-time students not pursuing a diploma, certificate or applied degree**)

Student Profile Data Compared to Survey Data:

As part of the administration for both the advanced and formal StudentSatisfaction and Engagement Surveys, the Supplier is required to gather studentprofile data for each program of study that will be surveyed. A precise audit of theKPI responses cannot be done since the students will not be identified in theprocess. In lieu of an audit, the student profile data will be used to analyze thesurvey responses to ensure that they are representative of the studentpopulation.






Operating Procedure


Colleges will receive reports at the college level (numbers and percentage) comparing student profile data with the survey data in the following categories: gender, age, language, and student registration status.

Data File Structure and Documentation:

A data file structure is a structure that positions the data within field names andpositions. The data file of results is provided by the Supplier in ASCII file and dbfformat.

Margin of Error:

Each college will receive the Provincial and its own student sample statisticalmargin of error.






Operating Procedure


Appendix J: Review and Adjustment of the KPI Data

In the Fall of each year, the Ministry launches, with a memo to college presidents, the annual review and adjustment process on data harvested in the previous cycle.

The purpose of the review and adjustment process is to ensure that only data obtained from eligible students, graduates, and employers are used in the calculation of the KPI. This is particularly important for the three KPIs (graduate employment, graduate satisfaction, and employer satisfaction) that are used to distribute performance funding.

The student satisfaction data the college receives from the Supplier in March already incorporate adjustments and exclusions necessary to meet the requirements of the survey administration protocol. The following are not included in the KPI calculation and are not eligible for inclusion:

unsealed or incorrectly sealed envelopes; survey packages received after the final drop-off date; certification labels received without an individual’s name and/or signature; surveys that are returned loose; and surveys that are returned in non-standard envelopes.

All other surveys are included in the college KPI calculation.

Colleges can review the KPI summary report for the Student Satisfaction and Engagement Survey that was provided by the Supplier to each college along with the actual data. The report lists the condition of survey packages received, problems encountered, and packages that were excluded from the KPI calculation.

If there are other adjustments that the college would like the Ministry to consider for inclusion in the Student Satisfaction KPI, these must be forwarded to the Ministry by December 23, 2016. The Ministry reviews the adjustments that the college requested, and all approved adjustments are forwarded to the Supplier.

If no further adjustments are required by the college, the Certification of Student Satisfaction Key Performance Indicator form shown below is returned, with the president’s signature, by e-mail to the Postsecondary Accountability Branch director at [email protected] and mailed to the Director, Postsecondary Accountability Branch, Ministry of Training, Colleges and Universities, Mowat Block, 7th Floor, 900 Bay Street, Toronto, ON, M7A 1L2.







Operating Procedure


The Supplier incorporates all approved adjustments and produces a final report in January 2017. Each college is provided with a final KPI report that includes:

details of adjustments as a result of the review; final raw student satisfaction data; and final Student Satisfaction KPI results.

Click on “Certification of Student Satisfaction Key Performance Indicator” form as a data entry form.

You can click on the link above to get an electronic version of this form.

You can fill in all the fields on your computer except the president’s signature.






Operating Procedure


You should then print it out and have your president sign it before sending it to the Ministry.

Appendix K: College Codes

CAATs are to use the college codes listed below when submitting files to the Supplier.

College Name Code Algonquin College ALGO Collège Boréal BORE Cambrian College CAMB Canadore College CANA Centennial College CENT Confederation College CONF Conestoga College CONS Durham College DURH Fanshawe College FANS Georgian College GEOR George Brown College GRBR Humber College HUMB La Cité Collégiale LACI Lambton College LAMB Loyalist College LOYT Mohawk College MOHA Niagara College NIAG Northern College NORT Sault College SAUL Seneca College SENE Sheridan College SHER St. Lawrence College SLAW Sir Sandford Fleming College SSFL St. Clair College STCL

Other Public Institutions (OPIs) are to use codes listed below when submitting files to the Supplier.

Institution Name Code Michener Institute for Applied Health Science MICH






Operating Procedure


Niagara Parks Commission School of Horticulture NPAR Ridgetown Campus – University of Guelph RIDG

Appendix L: Non-traditional Pattern of Program Delivery

Examples of Non-traditional pattern of program delivery may include:

Advanced June Survey

students enrolled in a one-year program - winter (semester 1) in-school, andsummer (semester 2) in-school

students enrolled in a one-year program - fall (semester 1) in-school, winter -co-op work term, and summer (semester 2) in-school

students enrolled in two or three-year programs - summer (graduating semester4 or 6) in-school

Advanced November Survey

students enrolled in a one-year program - summer (semester 1) in-school, andfall (semester 2) in-school

students enrolled in a one-year program - winter (semester 1) in-school,summer - co-op work term, and fall (semester 2) in-school

students enrolled in two or three-year programs - summer - co-op work term, andfall (graduating semester 4 or 6) in-school

RIDG






Operating Procedure


Appendix M: Surveying Apprentices in 2015-16

During the 2015-16 Student Satisfaction and Engagement Survey cycle, colleges are required to utilize the KPI Student Satisfaction and Engagement Survey to survey in-class apprentices. Colleges are asked to make the survey available to all apprentices attending all full-time block classes and approximately fifty per cent part-time classes, susubbjectject toto lo loccaal pl progrograramm d deelilivveerryy m mooddeess a anndd schscheedduules.les.

Colleges may choose which campuses, trades, in-class levels and sessions they survey. Once a class has been selected, all students in attendance are to be provided with an opportunity to complete the survey.

Colleges may choose to include Ontario Youth Apprenticeship Program (OYAP) classes.

Reportable subject-based classes of which the duration is thirty days or more are in the scope of the 2015-16 survey as part-time classes. Reportable subject-based classes refer to the classes that do not complete all reportable subjects of approved in-school training curriculum for one level. These classes vary greatly in duration, from several hhoouurs trs too mmoore tre thhaann 4 400 ddaayys.s.

Five-day full-time classes such as Exam Preparation Support classes are not included in the scope of the survey.

The survey administration process for apprentices is to follow the existing KPI Student Satisfaction and Engagement Survey Operating Procedure with the exception that:






Operating Procedure


a. the survey is to be administered to in-class apprentices in their final two in-classweek of any level.

b. colleges are to return the apprentice surveys to the service provider via courieror mail, either as completed, or once a month. Colleges are to notify the serviceprovider by email each time that a package of surveys has been sent. The lastday to survey is February 19, 2016, which is one week after the last day of theformal Student Satisfaction and Engagement Survey. The last day to submitpackages is February 24, 2016, which is one week after the last day to submitthe formal Student Satisfaction and Engagement Surveys.

c. colleges are to include apprentices, as a separate file, in their college profiledata.

Colleges must ensure full integrity of the KPI Student Satisfaction and Engagement Survey process through the secure storage of blank and completed surveys.

Colleges will receive their own apprentice survey data.

Apprentice survey results will not be included in the overall Student Satisfaction KPI.

(TBD: The OCAS KPI data warehouse will not be provided with apprentice survey )

Apprenticeship Programs of Instruction File

FIELDS Length (characters)

Position

1 College Code (e.g. HUMB) 4 1-4

2 Campus or location (e.g., Newnham) 30 5–34

3 Campus code (i.e., short form or abbreviation of campus or location, e.g., NM) 8 35–42

4 College program name (e.g., Law & Security) 45 43–87

5 College program name (long version for some reports) 75 88-162


12 163-174

7 Approved program sequence (APS) number 5 175-179

8 MTCU code 5 180-184






Operating Procedure


9 College division/school/faculty name (optional) 45 185-229

10 College division/school/faculty code (optional) 8 230-237

11 -DIGIT alphanumeric program code (optional) 3 238-240

12 Campus location postal code (6 characters) 6 241-246

13 Apprenticeship (Y/N) 1 247

14 Class (numeric) 7 248-254

15 Trade (alpha-numeric – 4 or 5 characters) 5 255-259

16 Full-Time/Part Time (F/P) 1 260

17 Level (1, 2 , 3 ) 1 261

18 Session (1, 2, 3 or 4) 1 262

Notes: Leave fields 7 (APS) and 8 (MTCU) blank for apprenticeship

Apprenticeship Student Profile Data Record Layout

Student Profile Data Record Layout (Apprenticeship) Fields Length (characters) Position


2. -DIGIT CODE assigned by the Supplier or the college 3 5–7

3. College program name (e.g., Law & Security) 45 8–52 4. College program name (long version for some reports) 75 53-127


12 128-139

6. Trade (alpha-numeric – 4 or 5 characters) 5 140-144

7. TBD5 145-149

8. Campus or location (e.g., Newnham) 30 150-1799. Campus code (i.e., short form or abbreviation of campus or

location, e.g., NM)8 180-187

10. Program TOTAL enrolment number enrolled in in-class 3 188-190

or 4

THREE

THREE2.






Operating Procedure


apprenticeship training that is funded by MTCU through the Seat Purchase Program. (

Program full-time enrolment enrolled in full-time block in-class apprenticeship training that is funded by MTCU through the Seat Purchase Program.

3 191-193

12. Program part-time enrolment number enrolled in part-time in-class apprenticeship training that is funded by MTCU through the Seat Purchase Program.

3 194-196

13. Number of males in target group 3 197-199

14. Number of females in target group 3 200-20215. Number with English as a first language in target group 3 203-20516. Number with French as a first language in target group 3 206-20817. Number with Other as a first language in target group 3 209-21118. Number ages less than 21 in target group 3 212-214



21. Number ages 31–35 in target group 3 221-22322. Number ages more than 35 in target group 3 224-226


Field Notes

10 to 22. Complete a student profile data record for each in-class apprenticeship

training programs that is funded by MTCU through the Seat Purchase Program.

10 to 22. Complete a student profile data record for each in-class apprenticeship

training programs that is funded by MTCU through the Seat Purchase Program.

Documents

KPI Student Satisfaction and Engagement Survey 2015 · PDF fileKPI Student Satisfaction and Engagement Survey 2015-16 Survey Cycle Ministry of Training, Colleges and Universities