1. Queenie CC Lam Hong Kong Observatory 8 December 2011
Sharing of Experience from Hong Kong, China on the Implementation of AMP Competency Assessment System
Queenie CC Lam
Hong Kong Observatory
8 December 2011
WMO VCP Workshop on Implementing Competency Assessment for Aeronautical Meteorological Personnel
Hong Kong, China, 5-9 Deceomber 2011

2. Outline
Implementation plan of Competency Assessment System (CAS) for Aeronautical Meteorological Observers (AMOB) and Aeronautical Meteorological Forecasters (AMF)
Implementation steps and tools used
Feedback from assessors and assessees

3. HKO’s Implementation plan of CAS
Oct Gap Analysis
Dec 2010 – Jan System and Tools Design
Feb Refresher Training Courses &
Briefing on CAS for regular Aviation Forecasters & Observers
Mar CAS Launch
Assessment for regular Aviation Forecasters & Observers
Apr-May CAS assessment summary and follow-up action ;
System review and refinement
Jun Briefing on CAS for non-regular Aviation Forecasters & Observers
Aug-Oct Assessment for non-regular Aviation Forecasters & Observers
Nov CAS assessment summary, system review and refinement
Integration of CAS into QMS

4. Implementation Steps Initial Review System Design Feedback
Preparation of
Documents
System
Implementation
System
Review

5. Implementation steps and tools used – Initial review
Scope and Target Assessees
In the CAS-AMOB, the target assessees for assessment include :
All Weather Observers
Scientific Assistants who are trained Weather Observers and may take up weather observation duties at the Airport Meteorological Office occasionally
In the CAS-AMF, the target assessees for assessment include :
All regular and occasional Aviation Forecasters
Experimental Officers who are trained Aviation Forecasters and may take up weather forecasting duties at the Airport Meteorological Office occasionally

6. Responsibilities Top Management Officer-in-Charge
Lead-Assessor of CAS-AMOB
Lead-Assessor of CAS-AMF

7. Initial Review – Gap Analysis
Gap analysis using cross-reference table method
Knowledge and skills required
Source of knowledge & acquisition of skills
Trainin gap
Remarks
Lectures
Read-up materials 1 2 3 .

8. Initial Review – Gap Analysis (2)
HKO’s gap analysis cross-reference table for AMOB

12. Initial Review – Gap Analysis (3)
HKO’s gap analysis summary for AMOB

13. Initial Review - Refresher Training
Organise refresher training courses to address the knowledge gap

14. HKO’s Competency Assessment System (CAS) Overview
System Design
HKO’s Competency Assessment System (CAS) Overview

15. System Design – Assessment Tools
Direct observation
Oral/Experiential questions
Written assessment
Case study/simulation

16. System Design – Competency Assessment Matrix for AMOB

17. System Design – Competency Assessment Matrix for AMF (1)

18. System Design – Competency Assessment Matrix for AMF (2)

19. System Design – Selection of Assessors
Staff responsible for the supervision of AMP and selected trainer(s) of the Weather Observation Course and Meteorology Course for Aviation Forecasters as appropriate
satisfy the qualification criteria for a trainer on the subject as specified in Quality System Procedure on training
Preferably possess knowledge of internal audit
For assessors of AMF – qualified as Meteorologist following WMO requirements

20. System Design – Officers
Lead-Assessor
prepare and conduct assessments in coordination with other assessors
recommend overall competency rating of AMP
Maintain individual AMP portfolio
Officer-in-Charge, Division Head of Aviation Weather Forecast and Warning Services
oversee the effective operation of the CAS
endorse the competency rating (either “Competent” or “Not Competent”)
Top Management, Assistant Director of the Aviation Weather Services Branch
approve the structure and implementation plan of the CAS
ensure the integrity of CAS as part of the QMS

21. System Design – Schedule & Validity
Assess every two years (similar to the frequency of revalidation test for ATC controllers in Hong Kong)
Guidelines for determining the schedule
Allow flexibility to cater for different types of weather situations in different seasons
Last around 1 day for each assessee
Agreed between assessor and assessee
Competence notification will carry a validity period of 24 months (in consistent with the assessment frequency)
revalidation of competence notification for another 24 months before expiry

22. System Design – Staff Engagement
Comments and engagement from assessors and assessees

23. Preparation of Documents - Checklists
Example : a sample of HKO’s checklists for Aeronautical Meteorological Observers

24. Preparation of Documents – Assessment Papers
Around 30 multiple choice questions
Around 10 short questions
Time allowed 2 hours
Passing mark 70

25. Preparation of Documents – Case Study/Simulation

26. Preparation of Documents – Assessment Procedures
Prior to assessment – prepare for assessment
During assessment – conduct and record assessment
Post assessment – feedback, document, evaluate, review and follow-up
Reassessment procedures

27. System Implementation
set implementation date
Conduct briefings to assessors and assessees
arrange assessment dates
Carry out assessment process using planned tools based on competency assessment matrix
Collect evidence and produce records
Keep records of training, qualification, years of forecasting experience, competency assessments in the Competency Portfolio of individual AMP

28. System Review (1) Implementation plan on schedule
Feedback/Suggestions from assessors and assessees and observations
Observers see the assessment as a good training and learning mechanism
facilitate continuous improvement
suggest regular refresher courses
Suggest to allow more time for preparation by assessees in between refresher training courses and assessments
Suggest to record answers in softcopy form when taking written assessments especially the short questions to save time

29. System Review (2)
Minimal oral questions during direct observation assessment to minimise interference to operational work of the assesses
Combined testing of competency performance criteria as far as practicable
Make use of routine weather briefing (such as weather conference/teleconference, handover briefing, briefing to clients) to evaluate the assessee by direct observation
Refine CAS based on feedback/suggestions and observations
Implement suggestions made in the review

30. Sharing of Implementation Plans
Webpage on CAT developed by CAeM TT-CAT
on the CAeM training website (
“CAT” under “Regulatory and Reference Material”
Login as a guest

31. Q&A