Note - sqa.org.uk CMV report 2013-14 POST... · Web viewRestricted or commercially sensitive information gathered during SQA Accreditation’s quality assurance activities is treated

Centre Monitoring ReportSafety Training Awards (STA)

15 August to 27 August 2013

Centre Monitoring Report Safety Training Awards: August 2013

NoteRestricted or commercially sensitive information gathered during SQA Accreditation’s quality assurance activities is treated in the strictest confidence. However, please note the following:

The findings of this report and the associated Action Plan will be presented to SQA’s Accreditation Committee.

The report and Action Plan will be published on SQA Accreditation’s website following receipt of the signed acceptance of audit findings.

The contents will contribute towards the Quality Enhancement Rating which will, in turn, contribute towards the quality assurance activity and timescales.

Please note that SQA Accreditation’s quality assurance activities are conducted on a sampling basis. Consequently, not all aspects of an awarding body’s performance in quality assurance, contract compliance, implementation, awarding of certificates and fee arrangements (not an exhaustive list) may have been considered in this report to the same depth.

© Scottish Qualifications Authority 2013 (v1)


Contents

1 Introduction 1

1.1 Scope and approach of centre monitoring 1

1.2 Centre monitoring report timeline 2

1.3 Centre monitoring dates 2

1.4 Overview 2

2 Centre monitoring findings 3

2.1 Areas of good practice 3

2.2 Requirements 3

2.2 Recommendations 8

3 List of documents reviewed during monitoring 9

4 Risk rating of Requirements 12

5 Action Plan 13

6 Acceptance of centre monitoring findings 19

© Scottish Qualifications Authority 2013 (v1)


1 Introduction

1.1 Scope and approach of centre monitoringSQA Accreditation conducts quality assurance activities of all awarding bodies offering SQA accredited qualifications or Units. This involves monitoring a sample of the awarding body’s approved centres/providers or assessment sites. All centre monitoring will be conducted in a consistent manner within and between centres. The aim of monitoring is to:

Ensure compliance under SQA Accreditation’s Regulatory Principles (2011), Regulatory Principles Directives, the requirements of the clauses within and any conditions attached to the approved awarding body agreement and the Criteria for Accredited Qualifications.

Confirm that quality assurance arrangements are being conducted by the awarding body in accordance with its prescribed arrangements.

Ensure that quality assurance arrangements are being conducted in a consistent manner, within and between centres.

Inform future audit and monitoring activity for the awarding body.

All Principles were included within the scope of the monitoring activity.

A Requirement has been raised where SQA Accreditation found evidence that the awarding body has not met SQA Accreditation’s regulatory requirements.

The following timescales apply:

SQA Accreditation will issue this report within 30 working days of the final centre monitoring date.

The awarding body must sign and return the report and associated Action Plan within 30 working days of the centre monitoring report being issued.

Within a further 20 working days of receiving the proposed Action Plan, SQA Accreditation will confirm whether the Action Plan is appropriate to address the Requirements. This will be subject to the actions proving appropriate to the Requirements raised.

SQA Accreditation will monitor progress towards completion of the actions identified in the Action Plan.

A Recommendation may be recorded in instances where SQA Accreditation considers there to be scope for improvement. Where these are agreed during centre monitoring, they are recorded on the report for future reference. As Recommendations are recorded for awarding body consideration only, it is not necessary to agree either actions or timescales to resolve these in the awarding body Action Plan.

© Scottish Qualifications Authority 2013 (v1) 1


1.2 Centre monitoring report timelineSQA Accreditation centre monitoring report date 18 September 2013

Date centre monitoring report and Action Plan to be signed

and submitted by Safety Training Awards 30 October 2013

1.3 Centre monitoring datesFour providers were monitored between 15 and 27 August 2013.

1.4 Overview As a result of the monitoring activities, four Requirements have been raised and two Recommendations have been recorded.

The four Requirements form the basis of the Safety Training Awards Action Plan. This must be completed and submitted to SQA Accreditation for agreement within 30 working days of the monitoring report being issued. The Action Plan must be submitted by 30 October 2013.

Outcome(s) Area(s) of concern Risk rating

Requirement 1 Principles 3, 6 and 22 High

Requirement 2 Principle 3 High

Requirement 3 Principles 6 and 7 High

Requirement 4 Principles 3 and 12 High

Recommendation 1 Principle 2 N/A

Recommendation 2 Principle 6 N/A



2 Centre monitoring findings

The following sections detail Requirements raised and Recommendations recorded against SQA Accreditation’s Regulatory Principles (2011), Regulatory Principles Directives, the requirements of the clauses within and any conditions attached to the Approved Awarding Body agreement and the Criteria for Accredited Qualifications.

2.1 Areas of good practiceThe following areas of good practice were noted:

The Course Organiser at Provider 1 highlighted the new STA Online system as being simple, straightforward and time saving.

The Course Organiser at Provider 2 commended the knowledge and pleasant manner of the STA customer service staff.

The Course Organiser at Provider 3 highlighted the prompt dispatch of course materials by STA and the quick resolution of issues on contact with the main office in Walsall.

The Course Organiser at Provider 4 commended STA on the high standards of moderation personnel and the quality of course content, especially the STA Professional Award in Teaching Swimming at SCQF Level 6.

2.2 RequirementsPrinciple 3. The awarding body must ensure that they employ robust processes to protect their own business interests as well as the interests of their approved centres and learners.

Principle 6. The awarding body and their approved centres must have the relevant expertise, quality assurance procedures, technological, financial, human resources and other physical resources, to carry out their regulated functions, during the life of the qualifications and Units they offer.

Principle 22. The awarding body must ensure that qualification and Unit certificates meet SQA Accreditation’s minimum requirements, reflect learner achievement, are only issued on the basis of a valid claim, and are supplied within a reasonable and clearly communicated timescale.

Candidates are required to be registered with STA after the first course session. In June of this year, the Course Organiser at Provider 2 tried to register a candidate on two courses, STA Professional Award for Pool Lifeguard at SCQF Level 7 and STA Award in First Aid at Work at SCQF Level 6, using the recently launched STA Online system.

However, STA Online would not allow the candidate to be registered as the candidate had been registered previously with STA for other qualifications. Even when the Course



Organiser entered the STA candidate number, obtained after contacting STA, the system still did not allow the candidate to be registered.

Both the candidate application form and registration fee were accepted by STA; the candidate completed the course and received a provisional results confirmation. However, at the time of the monitoring visit in August, the Course Organiser had still not managed to register the candidate despite ongoing discussions with the awarding body’s technical department. Consequently, the candidate had not received a full certificate. At the time of report writing, this issue had just been resolved.

STA must review STA Online and make any modifications necessary in order that candidates with existing STA records, can be registered and certificated successfully for subsequent qualifications.

The Course Organiser at Provider 3 uses the paper based system for registration and certification. When looking through documentation at this Provider, the Auditor found that the Course Acceptance Forms for three courses organised in 2012 and 2013 for Pool Emergency Procedures, First Aid at Work and Emergency First Aid at Work, showed qualification titles appended with ‘QCF’, denoting regulation by another regulator.

The Course Organiser had not noticed that the qualifications were not the correct SQA accredited provision and in fact was unaware of the difference between the regulators and their respective qualifications. On the basis of the information included in the forms, the Auditor would think it likely that the candidates on these three courses would have received certificates showing qualification titles relevant to the QCF provision.

During the visit, the Auditor also noted that two other Course Acceptance Forms for Emergency First Aid at Work did not include either QCF or SCQF in the qualification titles.

The Auditor is aware that Course Organisers must tick the appropriate regulator when completing a paper based Course Application Form. However to protect the integrity of SQA accredited qualifications and to safeguard the interests of candidates, STA must ensure that the correct regulator has been selected when processing paper based Course Acceptance Forms.

The issue of candidates being registered for QCF courses has been raised previously: a non-compliance in the STA Audit of October 2012 and as an observation in the monitoring report of 2012–13. As this issue persists and previous actions have not rectified this matter, it is being raised within the context of a Requirement in this monitoring report.

The Auditor tested STA Online for course registration and found that even selecting SQA as the regulator, still leads to courses which are all entitled as QCF provision, although the correct SQA accredited qualification does then appear within “My Courses” on the next screen. Clearly however, the titling needs to be addressed to ensure this process is accurate. In addition, within STA Online, SQA accredited qualifications are not fully represented as the manuals available to purchase have QCF titles, as do the course titles within the Document Store. Further, the course data sheets available in the downloads section of the STA website are specific to QCF titled qualifications, although the correct data sheets are available in other sections of the STA website.



The evidence available indicates that STA does not meet the requirements of Principles 3, 6 and 22. This has been raised as Requirement 1.

Principle 3. The awarding body must ensure that they employ robust processes to protect their own business interests as well as the interests of their approved centres and learners.

The STA Course Administration Procedures, 2012, Version 12.1 state that, “Tutors are required to retain the following course records for a period of three years from the course start date: Course Registration Form (CREG)”. The Auditor found that Course Organisers at Providers 1 and 3 had not retained the CREG forms.

The Procedures also state that, “it is an obligation on Course Organisers and Tutors to avoid the possibility of fraud by candidate impersonation, they must ensure that the identity of candidates is properly checked”. The Auditor found that the Course Organiser at Provider 3 did not check the identity of candidates, even though these candidates were not known to the Organiser personally.

The Procedures further state that, “Examination Papers must be returned using Recorded Delivery.” Course Organisers at Providers 2 and 3 did not use recorded delivery to return examination papers and indeed Provider 2 commented that examination papers had gone missing in the post several times.

STA must remind Course Organisers about the importance of adhering to the Procedures relevant to the retention of CREG forms, undertaking identity checks and the use of recorded delivery.

The evidence available indicates that STA does not meet the requirements of Principle 3. This has been raised as Requirement 2.




Principle 7: The awarding body must ensure that SQA Accreditation is granted access to the awarding body, their approved centres, assessment locations, staff, learners, premises, meetings, documents, data, analysis and evaluations on request.

A non-compliance relating to the STA moderation system was raised during the STA Audit of October 2012. As a result, the Moderation of STA Tutors was produced to clarify the process. The latter states that, “All STA Tutors are required to be moderated every 3 years both when tutoring and examining a course.”

The Tutor/Examiner in Provider 3 had evidence of moderation as an Examiner at three-yearly intervals but did not have any Tutor Moderation Forms to indicate that tutor moderation had taken place. The Tutor/Examiner could not recall being moderated as a Tutor in recent years and hence did not have any documentation to validate this activity.

STA must ensure that the moderation cycle is adhered to strictly for both tutoring and examining and that Tutors/Examiners retain this documentation for regulatory purposes.

The evidence available indicates that STA does not meet the requirements of Principles 6 and 7. This has been raised as Requirement 3.



Principle 3. The awarding body must ensure that they employ robust processes to protect their own business interests as well as the interests of their approved centres and learners.

Principle 12: The awarding body must ensure that assessments are accessible and produce results that are valid, reliable, transparent and fair.

The STA Course Administration Procedures state that, “the completed examination papers must be returned immediately after the Theory Examination, together with any spare Theory Examination Papers.” Also, the STA Tutor Manual Foundation Unit, 2009, Page 30, states that, “Immediately following the completion of the course and any final assessments the Course Organiser should return to the STA: c) Any surplus examination papers or forms; it is not permitted to keep, or take copies, of any examination papers.”

The Course Organiser at Provider 1 had retained one copy of a recent examination paper; the Course Organiser at Provider 2 had retained a large number of copied examination papers across different SQA accredited qualifications. Provider 1 was unaware of the STA requirement, as was Provider 2 who further explained that retention had been crucial given that papers had been lost in the post over the years. The Auditor reminded Provider 2 of the STA requirement of returning examination papers via recorded delivery to eliminate this risk.

Given that STA uses a question bank to generate papers randomly, the existence of copied papers could compromise the validity of external assessment. The retention of examination papers was raised as a non-compliance in the monitoring report of 2012–13. As a result, STA has produced a ‘Notice to Course Organisers, Tutors and Examiners August 2013’ which details the unacceptable nature of copying examination papers, the possibility of spot-checks through unannounced visits and the related sanctions. Although this is a recent document, given the serious nature of this issue, STA must now monitor compliance across Course Organisers, Tutors and Examiners.

The evidence available indicates that STA does not meet the requirements of Principles 6 and 7. This has been raised as Requirement 4.



2.2 RecommendationsPrinciple 2. The awarding body must publish clear information on their products, services and associated charges and fees.

The STA Tutor Structure, Version 2, 2010, states that a requirement to become a Tutor for STA is to, “hold an in-date teaching qualification(s) specific to the subject area”. This does not make sense as it implies that individuals would have to be qualified as Tutors before actually undertaking the Tutoring course.

Communication from STA indicates that this statement is only pertinent to aquatic tutors who must hold the appropriate aquatic teaching qualification. To ensure clarity of information for prospective Tutors, the requirement within the STA Tutor Structure should be amended accordingly.

This has been recorded as Recommendation 1.


Course Organisers are required to submit a course timetable along with a Course Application Form. To suit the needs of candidates, the Course Organiser at Provider 2 delivers Pool Lifeguard courses over seven days rather than the standard five days and therefore has to amend the STA timetable template. The Organiser has found that STA Online has not allowed the amended timetable to be saved. As a result, every time the Organiser books a Pool Lifeguard course, the template has to be amended. This process could be more streamlined and less time consuming if STA Online allowed Organisers to save any amended course timetables.

The Auditor also noted that timetables being used in Providers 1, 3 and 4 were labelled as timetables for QCF courses. STA should amend the titling of timetables to make them equally applicable to SQA accredited qualifications.

This has been recorded as Recommendation 2.



3 List of documents reviewed during monitoring

Document title Date of issue Version number

STA First Aid Manual

STA Emergency First Aid at Work Manual

STA Pool Rescuer Manual

Swimming Teaching Resource Manual

STA Award for Pool Responder Portfolio Version 13.1

STA Award for Pool Lifeguard Portfolio Version 13.2

STA Award in Swimming Teaching Portfolio Version 13.2

STA Certificate in Swimming Teaching Portfolio Version 13.2

STA Complaints Procedure

STA Enquiries about Result and Appeals Procedure

STA Course data sheets

Course Organiser Application Form 2013 Version 13.1



Course Administration Procedures 2012 Version 12.1

Procedure for Recognition of STA Tutor/Examiner

Tutor Manual Foundation Unit June 2009 Version 1.7

Moderation of STA Tutors

STA Online

STA Swimming Teaching Assessment Methodology

STA Pool Rescue Qualifications Assessment Methodology

Assessment Principles for First Aid qualifications March 2013 Version 3

STA membership details

Tutor occupational competence certificates 201120122013

STA Tutor/Examiner approval certificates

CPD records

Tutor update certificates 2012

Online technical competency tests 2013

Tutor/Examiner list

Physical activity readiness questionnaire



STA Online candidate registration forms

Course application forms

Course acceptance forms

Course registration and declaration forms

Candidate application forms

Course timetables

Candidate induction and course information

Practical marking sheets

STA Invigilators’ instructions 2012 Version 12.1

STA provisional results confirmation

Examination results sheets

Course evaluation forms

Candidate certificates

STA Membership forms

STA Feedback form — Examining 2012 Version 12.1

Tutor moderation forms 2009; 2012; 2013

Facilities health, safety and welfare check sheets



4 Risk rating of RequirementsSQA Accreditation assigns a risk rating to each Requirement recorded as a result of awarding body quality assurance activity. The table below illustrates how the rating for a Requirement is assigned. A weighting is applied that depends on the risk identified and the possible impact on qualifications and/or the learner of failure to implement that Requirement.

The assignment of a risk rating allows an awarding body to assign their resources to areas which have been identified as having a major impact on the qualifications and/or the learner. The risk rating also allows SQA Accreditation to assign its resources to support awarding bodies in improving their performance.

Risk Impact of Requirements identified through quality assurance activity

Very Low

The Requirement has been identified as likely to cause minimal concern and would not threaten the integrity of the qualification or impact adversely on the learner. Any overall effect is likely to be small scale and/or localised, rather than widespread. The identified Requirement is unlikely to recur once resolved and no long lasting damage would be anticipated.

Low

The Requirement has been identified as low impact but is of sufficient importance to merit intervention, with a low threat to the systems or procedures associated with the qualification and/or impact on the learner. Disruption may not just be localised but more widespread and would possibly cause residual damage; however, this could be easily corrected without further consequence.

Medium

The Requirement has been identified as having the potential to damage the credibility of the qualification and/or be detrimental to the learner. There may be some impact to the systems or procedures that support the qualification or the operational effectiveness of the awarding body.

High

The Requirement has been identified as having a potentially high impact on the integrity and reliability of the qualification, or the effective operation of the awarding body as a whole, if corrective action is not quickly taken. There is a high probability that the qualification and/or learner will be negatively affected.

Very High

The Requirement has been identified as having a serious impact on the integrity and reliability of the qualification or the effective operation of the awarding body if corrective action is not immediately taken. There is a very high probability that the qualification and/or learner will be negatively affected.

In assigning a risk rating, each Requirement is considered on its own merit, taking account of the context in which it was identified.



5 Action PlanA separate document in Microsoft Word has been forwarded with this report.

Areas of concern

Requirement Risk rating Proposed action(Please include a description of your intended methodology and details of the evidence that will be provided.)

Target date for completion

Principles 3, 6, 22

Requirement 1

At Provider 2, STA Online would not allow a candidate to be registered as the candidate had been registered previously with STA for other qualifications. This delayed certification.

STA must review STA Online and make any modifications necessary in order that candidates with existing STA records, can be registered and certificated successfully for subsequent qualifications.

High



Areas of concern



At Provider 3, Course Acceptance Forms showed that candidates had been registered for qualifications with QCF titles. The Auditor found further discrepancies in qualification titling within STA Online and the STA website.

STA must ensure that the correct regulator has been selected when processing paper based Course Acceptance Forms and that qualification titling discrepancies are corrected on STA Online and the STA website.



Areas of concern



Principle 3 Requirement 2

Across the Providers sampled, Course Organisers did not consistently retain Course Registration Forms, conduct candidate identity checks or return examination papers by recorded delivery, as required by STA Course Administration Procedures.

STA must remind Course Organisers about the importance of adhering to the Procedures relevant to the retention of CREG forms, undertaking identity checks and the use of recorded delivery.

High

Principles 6 Requirement 3 High



Areas of concern



and 7 The Tutor/Examiner in Provider 3 could not recall being moderated as a Tutor in recent years and hence did not have any Tutor Moderation Forms to validate this activity.

STA must ensure that the moderation cycle is adhered to strictly for both tutoring and examining and that Tutors/Examiners retain this documentation for regulatory purposes.

Principles 3 and 12

Requirement 4

Course Organisers at

High



Areas of concern



Providers 1 and 2 had retained copies of examination papers which contravenes instructions in the STA Course Administration Procedures and the STA Tutor Manual Foundation Unit.

STA recently issued a ‘Notice to Course Organisers, Tutors and Examiners August 2013’ with regard to the retention of examination papers. STA must now monitor compliance with this Notice across Course Organisers, Tutors and Examiners.



Signatures of agreement of action plan

For and on behalf of STA: For and on behalf of SQA Accreditation:

Signature

.................................................................

Signature

.................................................................

Date

.................................................................

Date

..................................................................



6 Acceptance of centre monitoring findings

For and on behalf of Safety Training Awards: For and on behalf of SQA Accreditation:

Signature

.................................................................

Signature

.................................................................

Designation

.................................................................

Designation

.................................................................

Date

.................................................................

Date

..................................................................




Documents

Note - sqa.org.uk CMV report 2013-14 POST... · Web viewRestricted or commercially sensitive information gathered during SQA Accreditation’s quality assurance activities is treated