D1.HRD.CL9.06 D1.HHR.CL8.07 D2.TRD.CL8.09 Slide 1

1.Determine the workplace training outcomes to be monitored and evaluated Slide 2 Obtain relevant planning documentation relating to workplace training provision 1.1 Clarify expected workplace training outcomes with relevant personnel 1.2 Determine specific workplace training outcomes identified by the organisation 1.3 Determine the specific workplace training outcomes to be monitored and evaluated 1.4 Schedule the workplace monitoring and evaluation process 1.5

1.1Obtain relevant planning documentation relating to workplace training provision Relevant documentation could include:  Business plans  Strategic plans  Specific training-related plans  Curriculum documentation  Competency standards and/or other relevant training criteria  Relevant organisation policies and procedures  Meeting minutes from planning meetings Slide 3

1.2Clarify expected workplace training outcomes with relevant personnel What type of training is delivered at your workplace? What kind of people are involved with training at your organisation? Slide 4

Relevant personnel connected to workplace training activities  Trainers and assessors  Administration staff and support staff  Specialist staff and subject/skills experts  Consultants  Supervisors and managers  Human resource staff  External training providers and representatives involved in the training delivery and/or assessment  Co-workers who have worked with staff who have received training  Current and past trainees  Other organisational stakeholders such as funding body representatives Slide 5

Determine specific workplace training outcomes identified by the organisation In any training activity we are trying to bring about change in three main areas. Trainers refer to these as the:  Psychomotor  Cognitive  Affective areas Slide 6

Determine specific workplace training outcomes identified by the organisation cont’d Slide 7 SpecificObservableMeasurable

1.4Determine the specific workplace training outcomes to be monitored and evaluated Determining training outcomes to be monitored and evaluated requires a decision between:  Monitoring and evaluating all training outcomes at one time or  Monitoring and evaluating different aspects of the process over time or as part of a cyclical process Slide 8

1.4Determine the specific workplace training outcomes to be monitored and evaluated Determining which specific training outcomes to monitor and evaluate may also be dependent upon several factors such as:  Monitoring and evaluation plans  Stakeholder requests  Feedback relating to specific training outcomes  Availability of resources, time and budget  Access to people with required skills for conducting the review  Business performance issues or results  Complaints Slide 9

1.5Schedule the workplace monitoring and evaluation process  Monitoring is the process through which the organisation ensures that actual training activities conform to the planned and intended ones  Evaluation is the systematic process of collecting and analysing information in order to determine whether and to what degree the objectives of a program or project have been, or are being achieved Slide 10

1.5Schedule the workplace monitoring and evaluation process cont’d The following questions should be answered in planning for evaluation and monitoring scheduling:  Who will conduct monitoring and evaluation?  What are the criteria for monitoring and evaluation?  How will the information be collected?  How will the information be utilised for the effective implementation of programs?  How will the information be shared?  What effects will the monitoring and evaluation activities have on other operational activities?  When should the information and data collected be forwarded to appropriate people? Slide 11

2.Monitor the effectiveness of training outcomes Slide 12 Review data captured relating to outcomes of training 2.1 Meet with relevant personnel to discuss captured training data 2.2 Ensure training data being captured enables effective evaluation to occur 2.3 Generate independent data relating to training outcomes 2.4 Compare independent training data with captured training data 2.5 Revise the data capturing process and records, as appropriate 2.6

2.1Review data captured relating to outcomes of the training Data relating to training outcomes could be captured from many different areas within the organisation. It is important to review the data to:  ensure captured data reflects identified data needs  ensure correct documentation is used to record outcomes of training  ensure personnel responsible for capturing training data understand their role  ensure relevant documentation is being recorded as anticipated  check that records are stored correctly  make objective judgments relating to the validity of the documentation  undertake spot checks to determine the authenticity of the records being maintained Slide 13

2.2Meet with relevant personnel to discuss captured training data  Meeting with relevant personnel to discuss the gathered data is an important part of the monitoring and evaluating process.  It provides an opportunity for people involved in the workplace training process an opportunity to clarify data, ask questions, offer their own feedback or interpretation, and it allows relevant personnel the chance to make suggestions and recommendations for improvement. Slide 14

2.3Ensure training being captured enables effective evaluation to occur Slide 15

2.3Ensure training being captured enables effective evaluation to occur Slide 16 Data collected Evaluation criteria Appropriate Y/N Follow-up action required Assessment records 90% of participants achieve competency Some assessment records incomplete or not up-to-date Ask relevant personnel to complete assessment records

2.4Generate independent data relating to training outcomes Sometimes the information that is being collected during training activities may not be sufficient or relevant to check if the training outcomes are being achieved. If, after checking draft evaluation data, there seems to be a gap in relevant data related to the evaluation, then supplementary data gathering materials may need to be developed and implemented. Slide 17

2.4Generate independent data relating to training outcomes Slide 18 Data collectedEvaluation criteriaAppropriate Y/NFollow-up action required/Supplementary information required Assessment records90% of participants achieve competency No. Some assessment records incomplete or not up-to-date Ask relevant personnel to complete assessment records Training course participant evaluation forms All areas of the training activity will be included in evaluation form No. No identified questions relating to training venue or materials Evaluation form updated with training venue and materials included and sent to all previous participants for completion 3 rd Party competency observation reports of staff who have completed specific training Observation reports from 3 participants from all training courses will be completed 2 months after each training course No. One training course (Communicate effectively on the telephone) had no workplace observations by 3 rd party observers completed Organise 3 rd party observation from this course and include results in evaluation Competency records for trained staff Training participants assessed as competent or Not Yet Competent No. Personal observations of staff classified as competent revealed several staff unable to perform tasks that they were assessed as competent in

2.5Compare independent training data with captured training data All training and assessment programs and activities collect data before, during and after the training. The information could range from:  Participant personal details to results from assessment activities  Complaints, suggestions, and trainer feedback information  Regular evaluations and scheduled audits of the training outcomes implemented at different times. Slide 19

2.5Compare independent training data with captured training data cont’d There are many factors that will influence the validity and relevance of data that is collected and any monitoring and evaluation process must consider not only the data results but also:  How the data is documented  When it is captured  Who captures it  Does the data reflect evaluation and training outcome purposes  Does the independently captured data match the data captured by relevant personnel Slide 20

2.6Revise the data capturing process and records, as appropriate Ensuring that captured data is relevant, sufficient, and useful for evaluation purposes, is a critical factor in deciding what to do next. Once the data has been analysed and checked against independent data, there may be a need to revise the data capturing process and records. Slide 21

Some issues that may arise from the data capturing process could include:  Challenges in gathering the data: time, location, cost, compiling the data, not enough human resources, not enough data records available  Issues with the content: questionnaire questions, survey questions, too many questions or not enough, language challenges for understanding questions  Data doesn’t reflect evaluation questions or purpose Slide Revise the data capturing process and records, as appropriate cont’d

3.Evaluate the effectiveness of training outcomes Slide 23 Assess training outcomes against identified need 3.1 Determine the performance of the trainers and assessors 3.2 Determine the effectiveness of the resources provided to support the implementation of training and assessment 3.3 Identify positives and negatives in relation to the attainment of identified specific workplace training outcomes 3.4 Identify positives and negatives in relation to issues arising as a result of the monitoring and evaluation process 3.5 Identify new topics and/or targets for future training and assessment requirements 3.6 Determine information that can be used to benefit future training and assessment decisions within the organisation 3.7

3.1Assess training outcomes against identified need It is important for the organisation to assess training outcomes against identified needs, in order to gauge whether the training activities are achieving the training outcomes that they were designed for. Slide 24

3.1Assess training outcomes against identified need cont’d What is validation? Slide 25 Tools Were assessment tools and training materials easy to use from both the trainer and candidate viewpoint and did they enable the collection of valid, sufficient, current and authentic evidence? Procedures What procedures, documents, records, and data capturing techniques were used and were they consistent with requirements such as sufficient, valid, reliable, flexible, easy to use and apply, and fair? Decisions How were decisions made in relation to the candidate’s performance and were these decisions well-supported by the evidence collected?

3.1Assess training outcomes against identified need cont’d Slide 26 QuestionYesNo Did the training and assessment tools adequately reflect the intended training outcomes in terms of activities and evidence? Did they adhere to the technical principles of training and assessment as well as the rules of evidence? Were they easy to use for the trainer, assessor and the candidate? Can they be used with a diverse range of candidates and be adjusted where necessary to ensure flexibility and fairness in their use?

3.2Determine the performance of the trainers and assessors So what are we actually evaluating? Effectively this process looks at four main things: Slide 27 The session’s success The Trainer’s performance The candidate’s experiences The outcomes achieved

3.3Determine the effectiveness of the resources provided to support the implementation of training and assessment Evaluating the effectiveness of the resources provided to support the implementation of training and assessment is a critical step in determining overall success of training outcomes. Slide 28

3.4Identify positives and negatives in relation to the attainment of identified specific workplace training outcomes It is important to analyse the data and identify the positives and negatives in relation to the achievement of training goals as this information will provide the basis for identifying ways to improve the training and assessment activities within the organisation. Slide 29

3.5Identify positives and negatives in relation to issues arising as a result of the monitoring and evaluation process Issues that may arise from the monitoring and evaluation process could include:  Issues with staff or participant communication  Costs - either positive or negative  Timing and flexibility issues  Discovering that trainers and assessors have more or less skills than required  Identifying changes in industry practice, technology requirements or equipment failure or obsolescence Slide 30

3.6Identify new topics and/or targets for future training and assessment requirements Slide 31

3.7Determine information that can be used to benefit future training and assessment decisions within the organisation With so much data collected and information analysed it is necessary to determine information that can be used to improve future training and assessment decisions within the organisation. Identifying the situations that enhance training and assessment activities and communicating this before future plans are made can assist in the development of a more appropriate and effective system. Slide 32

4.Prepare a report on the effectiveness of workplace training outcomes Slide 33 Develop a formal report on the effectiveness of workplace training outcomes 4.1 Make recommendations for change regarding workplace training and assessment provision 4.2 Distribute the report 4.3 Obtain feedback on the report 4.4

4.1Develop a formal report on the effectiveness of workplace training outcomes The basic elements of an evaluation plan include:  Title page  Executive summary  Question overview  Intended use and users  Program description  Evaluation focus  Methods  Analysis and interpretation plan  Use, distribution of results, and recommendations Slide 34

4.2Make recommendations for change regarding workplace training and assessment provision Suggestions for change could include:  Whether to continue with the existing training and assessment system with minor alterations or to cease with the current system in favour of an external provider administering training and assessment  Making recommendations relating to staff resources, training, payment, costs, assessment content or administration procedures  Expanding the existing system or reducing the scope of the existing system  Recommending changes to location and timing of the training and assessment system  Changing course content and materials  Changes to support infrastructure  Changes to targeted trainees and participants Slide 35

4.3Distribute the report Some questions to ask about the potential audience(s) are the following:  Who is a priority?  What do they already know about the topic?  What is critical for them to know?  Where do they prefer to receive their information?  What is their preferred format?  What language level is appropriate?  Within what time frame are evaluation updates and reports necessary? Slide 36

4.3Distribute the report cont’d Slide 37 What do you want to communicate? To whom to you want to communicate? How do you want to communicate? Format(s)Channel(s)

4.4Obtain feedback on the report Slide 38