WMO VCP Workshop on Implementing Competency Assessment for Aeronautical Meteorological Personnel...

WMO VCP Workshop on Implementing Competency Assessment for Aeronautical Meteorological Personnel

Hong Kong, China, 5-9 Deceomber 2011

Sharing of Experience from Hong Kong, China on the Implementation of AMP Competency Assessment

System

Queenie CC LamHong Kong Observatory

8 December 2011

Outline Implementation plan of Competency Assessm

ent System (CAS) for Aeronautical Meteorological Observers (AMOB) and Aeronautical Meteorological Forecasters (AMF)

Implementation steps and tools used

Feedback from assessors and assessees

HKO’s Implementation plan of CAS

Oct 2010 Gap Analysis Dec 2010 – Jan 2011 System and Tools Design Feb 2011 Refresher Training Courses &

Briefing on CAS for regular Aviation Forecasters & Observers

Mar 2011 CAS LaunchAssessment for regular Aviation Forecasters &

Observers Apr-May 2011 CAS assessment summary and follow-up action ;

System review and refinement Jun 2011 Briefing on CAS for non-regular Aviation Forecasters

& Observers Aug-Oct 2011 Assessment for non-regular Aviation Forecasters &

Observers Nov 2011 CAS assessment summary, system review and

refinement 2012 Integration of CAS into QMS

Implementation StepsInitial

Review

SystemDesign

Preparation ofDocuments

System Implementation

System Review

Feedback

Implementation steps and tools used – Initial review Scope and Target Assessees

In the CAS-AMOB, the target assessees for assessment include :

All Weather Observers Scientific Assistants who are trained Weather Observers and m

ay take up weather observation duties at the Airport Meteorological Office occasionally

In the CAS-AMF, the target assessees for assessment include : All regular and occasional Aviation Forecasters Experimental Officers who are trained Aviation Forecasters

and may take up weather forecasting duties at the Airport Meteorological Office occasionally

Responsibilities

Top Management Officer-in-Charge Lead-Assessor of CAS-AMOB Lead-Assessor of CAS-AMF

Initial Review – Gap Analysis Gap analysis using cross-reference table method

Knowledge and skills required

Source of knowledge & acquisition of skills

Trainin gap Remarks

Lectures Read-up materials

1

2

3

.

.

.

Initial Review – Gap Analysis (2) HKO’s gap analysis cross-reference table for AMOB

Initial Review – Gap Analysis (3) HKO’s gap analysis summary for AMOB

Initial Review - Refresher Training Organise refresher training courses to address the knowledge ga

p

System DesignHKO’s Competency Assessment System (CAS) Overview

System Design – Assessment Tools Direct observation

Oral/Experiential questions

Written assessment

Case study/simulation

System Design – Competency Assessment Matrix for AMOB

System Design – Competency Assessment Matrix for AMF (1)

System Design – Competency Assessment Matrix for AMF (2)

System Design – Selection of Assessors Assessors

Staff responsible for the supervision of AMP and selected trainer(s) of the Weather Observation Course and Meteorology Course for Aviation Forecasters as appropriate

satisfy the qualification criteria for a trainer on the subject as specified in Quality System Procedure on training

Preferably possess knowledge of internal audit

For assessors of AMF – qualified as Meteorologist following WMO requirements

System Design – Officers

Lead-Assessor prepare and conduct assessments in coordination with other assessors recommend overall competency rating of AMP Maintain individual AMP portfolio

Officer-in-Charge, Division Head of Aviation Weather Forecast and Warning Services

oversee the effective operation of the CAS endorse the competency rating (either “Competent” or “Not Competent”)

Top Management, Assistant Director of the Aviation Weather Services Branch

approve the structure and implementation plan of the CAS ensure the integrity of CAS as part of the QMS

System Design – Schedule & Validity Assess every two years (similar to the frequency of revalidation t

est for ATC controllers in Hong Kong)

Guidelines for determining the schedule Allow flexibility to cater for different types of weather situations in di

fferent seasons Last around 1 day for each assessee Agreed between assessor and assessee

Competence notification will carry a validity period of 24 months (in consistent with the assessment frequency)

revalidation of competence notification for another 24 months before expiry

System Design – Staff Engagement Comments and eng

agement from assessors and assessees

Preparation of Documents - ChecklistsExample : a sample of HKO’s checklists for Aeronautical Meteorological Observers

Preparation of Documents – Assessment Papers Around 30 multiple choice questions

Around 10 short questions

Time allowed 2 hours

Passing mark 70

Preparation of Documents – Case Study/Simulation

Preparation of Documents – Assessment Procedures Prior to assessment – prepare for assessment

During assessment – conduct and record assessment

Post assessment – feedback, document, evaluate, review and follow-up

Reassessment procedures

System Implementation set implementation date

Conduct briefings to assessors and assessees

arrange assessment dates

Carry out assessment process using planned tools based on competency assessment matrix

Collect evidence and produce records

Keep records of training, qualification, years of forecasting experience, competency assessments in the Competency Portfolio of individual AMP

System Review (1) Implementation plan on schedule

Feedback/Suggestions from assessors and assessees and observations

Observers see the assessment as a good training and learning mechanism

facilitate continuous improvement

suggest regular refresher courses

Suggest to allow more time for preparation by assessees in between refresher training courses and assessments

Suggest to record answers in softcopy form when taking written assessments especially the short questions to save time

System Review (2) Minimal oral questions during direct observation assessment

to minimise interference to operational work of the assesses

Combined testing of competency performance criteria as far as practicable

Make use of routine weather briefing (such as weather conference/teleconference, handover briefing, briefing to clients) to evaluate the assessee by direct observation

Refine CAS based on feedback/suggestions and observations

Implement suggestions made in the review

Sharing of Implementation Plans Webpage on CAT developed by

CAeM TT-CATon the CAeM training website (http://www.caem.wmo.int/moodle)

“CAT” under “Regulatory and Reference Material”

Login as a guest

Q&A