Process Level Evaluation of Internal Productivity, Communication, and Collaboration SIGnetwork Grant Management PLC Anna Harms March 26, 2012

Agenda Review MiBLSi’s Value Added Work System (Logic Model) Three Levels of Evaluation Process Level Evaluation: What, Why and How Next Steps

Management/CoordinationManagement/Coordination Capital Resources PeopleMaterialsInformation Work Systems: Providing the MTSS practices and the supports for the these practices to take place successfully within schools and districts FinancialEvaluation Professional Learning Technical Assistance Political Regulatory/Legal Economy Geography Society/Culture Political Regulatory/Legal Economy Geography Society/Culture Environmental Factors Receivers Stakeholders /Funders Investments: Funding Visibility Political support Returns: Addressing critical issues (Discipline/Ethnicity) Addressing program directives (State Performance Plan) Consumers schools, districts, ISDs/RESAs Consumers schools, districts, ISDs/RESAs MiBLSi Value- Added Work System Valued MTSS Products/ Services Priority, Plan, Formative Assessment, Continuous Improvement, Risk Management Determine worth

MiBLSi Organizational Structure

Set up Evaluation Systems LevelDescriptionTools OrganizationalEvaluating impact of the RtI project Stakeholders Addressing critical issues (e.g., Discipline/Ethnicity) Addressing program directives (e.g., State Performance Plan) Consumers Valued results (e.g., Improved quality program, fidelity of implementation, successful student outcomes) Annual reports Measures of program quality Measures of fidelity of implementation Measures of student outcomes Survey of consumer satisfaction ProcessEvaluating operational productivity and efficiency of each unit within organization Project management tools FastTrack Basecamp, Manymoon Performer (worker) Evaluating worker productivity and efficiency Job models

POLL: Which level(s) does your project evaluation emphasize? Organizational Process Individual Worker All A Combination of the Above

Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors

Annual Master Plan

Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators

Sample Portion of FastTrack Schedule

Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators Projects are entered into Manymoon project management software and specific tasks are assigned Early in the school year (by mid August) and ongoing (at least monthly) Unit Coordinators

Manymoon: Project Management

Manymoon

Tasks

Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators Projects are entered into Manymoon project management software and specific tasks are assigned Early in the school year (by mid August) and ongoing (at least monthly) Unit Coordinators Provide a status update of work accomplished, project ability to complete future work, and find solutions to barriers QuarterlyUnit Coordinators, Program Team

Problem Solving with Unit Data Gather Unit Data / Problem Identification, Description Problem Analysis Plan Development Plan Implementation

Unit Status Update in GoogleDocs

Problem Identification / Description List the activities that were not accomplished on time. How close were we to the goal (days, weeks)? Does a type of task / project seem to be the problem? What is the impact of not having the task / problem completed on time (within the project and external to the project)?

Problem Analysis Were non-complete tasks / projects a priority to begin with? Was the goal realistic? Were resources properly allocated? Were unanticipated task demands added?

Plan Development Specific projects / tasks for which problem solving is needed Adjust timeline / due date Reallocate resources Remove other tasks Prioritize this work and demote other work

Plan Implementation Adjust tasks and timelines in FastTrack and Manymoon.

Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators Projects are entered into Manymoon project management software and specific tasks are assigned Early in the school year (by mid August) and ongoing (at least monthly) Unit Coordinators Provide a status update of work accomplished, project ability to complete future work, and find solutions to barriers QuarterlyUnit Coordinators, Program Team Conduct interviews to gather feedback on collaboration and communication within and between units Quarter 3 (by end of June)External Evaluator

Interviews Scheduled in April External Evaluator will be conducting a combination of individual and group interviews Will summarize data and help us use the feedback to build stronger internal structures

Next Steps Early August: Pull together all data from Use data to inform Master Plan Reflect on the evaluation process and plan for adjustments that will allow for greater efficiency and systems to support ongoing data-based decision making

Discussion Process level evaluation data is not typically required for grant reporting. Are you gathering this type of data? What ideas do you have making process level evaluation valuable?

Three Evaluation Systems LevelDescriptionTools OrganizationalEvaluating impact of the RtI project Stakeholders Addressing critical issues (e.g., Discipline/Ethnicity) Addressing program directives (e.g., State Performance Plan) Consumers Valued results (e.g., Improved quality program, fidelity of implementation, successful student outcomes) Annual reports Measures of program quality Measures of fidelity of implementation Measures of student outcomes Survey of consumer satisfaction ProcessEvaluating operational productivity and efficiency of each unit within organization Project management tools FastTrack Basecamp, Manymoon Performer (worker) Evaluating worker productivity and efficiency Job models October 23, 2012 Evaluators’ PLC

POLL: I gathered information from today’s call that I can use to inform grant management and evaluation for my state’s SPDG Strongly Agree Agree Disagree Strongly Disagree

Contact Information Anna Harms Evaluation and Research Coordinator Michigan’s Integrated Behavior and Learning Support Initiative (616)