Monitoring & Evaluation. Objective Learn the why, what and how-to approach to monitoring Review monitoring techniques and define the roles monitoring.

Slides:



Advertisements
Similar presentations
MONITORING OF SUBGRANTEES
Advertisements

Leon County Schools Performance Feedback Process August 2006 For more information
Site-Based Decision Making Campus Planning. Restructuring A process through which a district or school alters the pattern of its structures (vision, rules,
Goals-Based Evaluation (GBE)
The Aged Care Standards and Accreditation Agency Ltd Continuous Improvement in Residential Aged Care.
Process and Procedure Documentation. Agenda Why document processes and procedures? What is process and procedure documentation? Who creates and uses this.
In Europe, When you ask the VET stakeholders : What does Quality Assurance mean for VET system? You can get the following answer: Quality is not an absolute.
Quality Improvement/ Quality Assurance Amelia Broussard, PhD, RN, MPH Christopher Gibbs, JD, MPH.
Data Collection Six Sigma Foundations Continuous Improvement Training Six Sigma Foundations Continuous Improvement Training Six Sigma Simplicity.
Decision Making Tools for Strategic Planning 2014 Nonprofit Capacity Conference Margo Bailey, PhD April 21, 2014 Clarify your strategic plan hierarchy.
Management: Analysis and Decision Making
An Assessment Primer Fall 2007 Click here to begin.
Competitive Grant Program: Year 2 Meeting 2. SPECIAL DIABETES PROGRAM FOR INDIANS Competitive Grant Program: Year 2 Meeting 2 Data Quality Assurance Luohua.
Title I Needs Assessment and Program Evaluation
Evaluation. Practical Evaluation Michael Quinn Patton.
How to Write Goals and Objectives
Standards and Guidelines for Quality Assurance in the European
Continuous Quality Improvement (CQI)
Development of Competence Profile Quality managers in VET-institutions Project no: PL1-LEO This publication [communication] reflects the.
Quality Improvement Prepeared By Dr: Manal Moussa.
Steps in a Marketing Research Project
Conducting the IT Audit
Codex Guidelines for the Application of HACCP
REVIEW AND QUALITY CONTROL
Internal Auditing and Outsourcing
PROGRAMS MONITORING AND SUPERVISION
Staff Structure Support HCCA Special Interest Group New Regulations: A Strategy for Implementation Sharon Schmid Vice President, Compliance and.
ISTEP: Technology Field Research in Developing Communities Instructor: M. Bernardine Dias CAs: Sarah Belousov and Ermine Teves Spring 2009.
9 Closing the Project Teaching Strategies
Applying the Principles of Prior Learning Assessment Debra A. Dagavarian Diane Holtzman Dennis Fotia.
Martha Thurlow and Laurene Christensen National Center on Educational Outcomes CEC Preconvention Workshop #4 April 21, 2010.
Business Analysis and Essential Competencies
Verification: Quality Assurance in Assessment Verification is the main quality assurance process associated with assessment systems and practice - whether.
Copyright  2005 McGraw-Hill Australia Pty Ltd PPTs t/a Australian Human Resources Management by Jeremy Seward and Tim Dein Slides prepared by Michelle.
Environmental auditing
Understanding Meaning and Importance of Competency Based Assessment
1 Unit 1 Information for management. 2 Introduction Decision-making is the primary role of the management function. The manager’s decision will depend.
Observation technique and assessment measurements 1.
Implementing QI Projects Title I HIV Quality Management Program Case Management Providers Meeting May 26, 2005 Presented by Lynda A. O’Hanlon Title I HIV.
Monitoring & Evaluation: The concepts and meaning Day 9 Session 1.
SUB-MODULE 5. MEANS OF VERIFICATION RESULTS BASED LOGICAL FRAMEWORK TRAINING Quality Assurance and Results Department (ORQR.2)
Gulana Hajiyeva Environmental Specialist World Bank Moscow Safeguards Training, May 30 – June 1, 2012.
Programme Objectives Analyze the main components of a competency-based qualification system (e.g., Singapore Workforce Skills) Analyze the process and.
Paul Hardiman and Rob Brown SMMT IF Planning and organising an audit.
Middle Leadership Programme Day 1: The Effective Middle Leader.
Indicators to Measure Progress and Performance IWRM Training Course for the Mekong July 20-31, 2009.
Monitoring and Evaluation
Implementation and follow up Critically important but relatively neglected stages of EIA process Surveillance, monitoring, auditing, evaluation and other.
Copyright 2010, The World Bank Group. All Rights Reserved. Principles, criteria and methods Part 2 Quality management Produced in Collaboration between.
ISO DOCUMENTATION. ISO Environmental Management Systems2 Lesson Learning Goals At the end of this lesson you should be able to:  Name.
Data Report July Collect and analyze RtI data Determine effectiveness of RtI in South Dakota in Guide.
Project management Topic 1 Project management principles.
Measuring Results of Improvement Actions Márcio Rodrigues, Tallin, 13/01/2015.
Consistency of Assessment (Validation) Webinar – Part 1 Renae Guthridge WA Training Institute (WATI)
Evaluate Phase Pertemuan Matakuliah: A0774/Information Technology Capital Budgeting Tahun: 2009.
Assessment Design and its relationship to NARS and ILOs Arthur Brown Advisor to the Quality Assurance and Accreditation Project Republic of Egypt.
Assessment Validation. MORE THAN YOU IMAGINE ASQA (Australian Skills Quality Authority) New National Regulator ASQA as of 1 July, 2011.
Monitoring Afghanistan, 2015 Food Security and Agriculture Working Group – 9 December 2015.
Regional Dental Consultants’ Meeting Presented by Emerson Robinson, DDS, MPH Region II and V Dental Consultant.
Search Engine Optimization © HiTech Institute. All rights reserved. Slide 1 Click to edit Master title style What is Business Analysis Body of Knowledge?
Chapter Two Copyright © 2006 McGraw-Hill/Irwin The Marketing Research Process.
Session 6: Data Flow, Data Management, and Data Quality.
Observing and Assessing Young Children
P3 Business Analysis. 2 Section F: Project Management F1.The nature of projects F2. Building the Business Case F4. Planning,monitoring and controlling.
Chapter 10 Understanding and Planning Reports and Proposals 10-1.
Safety Management Systems Session Four Safety Promotion APTA Webinar June 9, 2016.
Selection Criteria and Invitational Priorities School Leadership Program U.S. Department of Education 2005.
Monitoring and Evaluation Systems for NARS Organisations in Papua New Guinea Day 3. Session 9. Periodic data collection methods.
Public School Monitoring Roadmap
Process and Procedure Documentation
Presentation transcript:

Monitoring & Evaluation

Objective Learn the why, what and how-to approach to monitoring Review monitoring techniques and define the roles monitoring plays in our lives both personally and professionally Understanding the types of monitoring and their impact on program operations Provide insight on the critical elements of monitoring Open Discussion

What Is Monitoring? Terminology Minefield Monitoring is a very widely used and ambiguous term Merriam Webster Monitoring is to watch, keep track of, or check for a special purpose General Definition Used to describe the process of checking or regulating some defined activity

Why Monitor?

Importance of Monitoring Through assessment, we know how well we are doing Through assessment, we know how effective and efficient we are Through assessment, we are able to monitor our resources You may go on using resources to no useful end

Why Monitor? Monitoring can be an invaluable management tool that enables you to assess the quality and impact of your work against your action plans

Why Monitor? Like riding a bicycle, you can adjust as you go along and ensure that you are on the right track

Why Monitor? Reasons Monitoring Is Not Popular It’s intrusive If it’s not broke, no need to fix it Other competing priorities No complaints, no problems Not necessary Like pulling teeth

Why Monitor? What If We Didn’t Monitor Our Health What We Eat Restaurants- Food Wardrobe Check-book Investments Pollution Homes Government Water Quality

Role of Monitoring Monitoring Is Used In… Project planning and implementation Analyzing the situation Determining whether the inputs in the project are well utilized Identifying problems facing the project or program and finding solutions

Role of Monitoring Monitoring Is Used To… Ensure all activities are carried out properly by the right people and in the allotted amount of time Apply lessons from one project experience on to other projects Determine whether project planning and problem solving is as appropriate and efficient as possible

Role Of Monitoring 4 Principles of Monitoring 1.Monitoring is the regular observation and recording of activities taking place in a project or program 2.Monitoring is a systematic and purposeful observation based on defined inputs 3.Monitoring involves giving feedback about the progress of the project or program 4.Monitoring enables the gathered information to be used in making decisions for improving project performance – Lessons learned

Role of Monitoring Monitoring Is A Feedback Loop  For Example Checking to ensure that we did what we said we would do Checking the assumptions and estimates used to make decisions Checking on program quality and efficiency Feedback and continuous improvement

Role Of Monitoring Monitoring Can Be Carried Out To meet one or more of a wide variety of objectives By measuring one or more of a wide range of variables At different levels of resolution

What To Monitor What Are We Monitoring Program Requirements Sensitivity and Risk Analysis Can determine key variables in developing a monitoring plan and monitoring requirements

World of Monitoring What? Why? How? Training Performance Implementation plans Certifications Effectiveness Efficiency Validation Impact Onsite or remotely Records Interviews

World of Monitoring Strategic Plan ImplementMonitor Reflect Learn Decide Adjust Feedback

Types Of Monitoring The phrase “monitoring is used in a wide variety of situations” Examples Include: Fiscal Monitoring Validation Monitoring Compliance Monitoring Effectiveness Monitoring Administrative Monitoring Programmatic Monitoring

Program Monitoring Programmatic Compliance Monitoring Consists of 2 Components Administrative Compliance Asks the question, “how well does RWB administrative documentation reflect actual conditions on the ground before and after program activities took place?” Field Compliance Asks the question, “how well are the rules and the terms of the program being applied on the ground?”

Program Monitoring Programmatic Compliance Monitoring The primary focus of programmatic monitoring is: To assess program operational practices, procedures and processes for compliance with applicable federal and State laws, policies and guidance To ensure that policies and practices relative to program quality and outcomes meet the program’s objectives To assist the RWB in identifying problem areas and recommending alternatives and practices that will promote positive practices and outcomes

Program Monitoring Critical Elements Define population of interest and take a representative sample Compare estimates from the representative sample with specific benchmarks Various levels of resolution depends on objectives and target population

Sampling Techniques Statistical Sampling - probability sampling where every item in the population under review has a known chance of selection Judgmental Sampling – sample size or composition determined by professional judgment rather than the laws of probability Sliding Scale - approximates the sample size determined using a semi-statistical approach Bootstrap Method?

Sampling Techniques DEO’s program monitoring section uses a statistical, random sampling method to select program files to be reviewed for each program

Sampling Techniques NO MATTER What methodology is used, make sure that it is a valid representative of the population being reviewed

Tools &Guides Types Of Tools/Guides Used: Process Management Review Tool Administrative Process Specific Review Tool Compliance

Tools & Guides Submission To Regions Monitoring tools are normally submitted to RWB staff at the beginning of the annual monitoring cycle for review and feedback

Tools & Guides Process Management Review Tool Asks questions about program processes, design elements, and organization Allows you to become familiar with program operations Evaluates program design in the context of whether it is demand-driven Allows you to draw objective conclusions about program operations, including promising “Best Practices”

Tools & Guides Program Specific Tool Used to determine compliance with specific program requirements Serves as an indicator of whether objectives or requirements are being met Identifies issues that may be contrary to law or other requirements

Tools & Guides A separate tool is developed for each program monitored Areas covered by tools include questions related to: Training Services Support Services Credential Attainment Participant Eligibility Criteria Assessment and Plan Development Grievance/Complaint/EEO requirements

Local Initiatives Quality Assurance Establish goals and objectives on what you want accomplished Develop an action/implementation plan based on the goals, activity levels, targets, etc Develop local operating policies/procedures/guidelines based on core management functions and program operations Develop tools based on your objectives, rules, tenets of law and other guidance (one or more indicators to assist in determining if objectives have been met) Train staff

Local Initiatives Case Managers Ensure all files have required documentation Develop a checklist of what should be included in the files Make sure detailed case notes are entered in system Conduct peer reviews/evaluations Exchange case files for review by different case managers Check accuracy of source documents in files against system data

Summary Monitoring is a very ambiguous term that is used for continuous improvement Don’t worry about what a monitoring program is called Focus on The pre-defined objectives What is being done Why it’s being done What impact is it having Remember that monitoring is the feedback loop that can provide useful information in planning and making changes

References Phil Bartle, PhD Civicus J.S Thrower & Associates Ltd.

Questions? For More Information On Monitoring and Evaluations Please Contact Ken Williams at lorida.com