1. Welcome to the SPDG Webinar
Measuring Levels of Collaboration
David Merves, Evergreen Evaluation & Consulting, Inc.
The 2014 Program Measures Results
Jennifer Coffey, PhD, SPDG Program Lead 1

2. Webinar Ground Rules Mute your phones:
To Mute and Un-mute your phone line, Press *6
Please do not put your phones on ‘Hold’
For webinar technical difficulties:
Send a chat message to “Audrey” or “Jennifer” or send to
How to ask questions (audio/chat):
- Audio/Voice: During the Q&A, un-mute your phone *6
-Chat Pod: Type your question in the chat pod
Archive Recording
- To be posted to 2

3. For program measure guidance, please join us Tuesday, December 9th from 3:30-5:00 ET
Jennifer Coffey will be sharing information about:
• How to respond to each program measure in your upcoming APR.
• An updated Worksheet for Program Measure 1.
• Lessons learned from the 2014 review.
Number: 1 (877) Passcode: 3

4. Upcoming Bidders’ Webinars
Jennifer Coffey will be presenting information regarding the SPDG application process for Fiscal Year 2015 and will answer questions.
1st Bidders’ Webinar
December 15th 3:00-4:30 ET
Teleconference: 1 (877) Passcode: (to be dialed in by all participants and presenters)
2nd Bidders’ Webinar
December 22nd 3:00-4:30 ET 4

5. SAVE THE DATE! Continuation Reporting (Annual Performance Report) Guidance
Putting together your APR
& Learning from others
Date: January 8, Time: 3:00-4:30pm ET
Number: 1 (877) Passcode:
In preparation for the calls, please review the Program Measures Webinar from December 9th. 5

6. 2014 SPDG Program Measures Evaluation
How we improved
GPRA Outcomes
Areas that need improvement
What you can do
What can we (OSEP/SIGnetwork) do?

7. How we improved Provided good information for Measure 1
Better descriptions of fidelity measures
Use of the correct equation for Measure 2

8. GPRA Outcomes Program Measure 1 Met Target Data Not Available1 Yes No
Met Target
Data Not Available1
Yes No
Year two
(40 initiatives) 1
Year three
(13 initiatives)
Year four
(14 initiatives)
Total
(67 initiatives, 37 states) 3 % 4%

9. Program Measure 1
2014: In conversation…
2013: Of 33 project initiatives, 15 initiatives met the benchmarks (45%)
Areas of greatest need:
A2 -Clear expectations are provided for PD participants
D1 - Accountability for fidelity measurement/reporting system is clear
D4- Goals are created with benchmarks for implementation and student outcome data, and plans are in place to share and celebrate successes.
E2-Leadership analyzes feedback from staff and makes changes to alleviate barriers and facilitate implementation, including revising policies and procedures to support new way of work.

10. Program Measure 1: Problems with Reporting
Need to report on each initiative separately
Be clear whether activities have already been conducted or not
Include your worksheet with your APR

11. For program measure 1: How can we help?
What would help you in meeting targets for evidence-based professional development?
What has worked for you in the past to improve your components?
What are barriers for you?

12. Program Measure 2: Current Year
Met Target
Data Not Available
Unable to Determine
Yes No
Year three
(13 initiatives) 5 1 2
Year four
(14 initiatives) 7
Total
(27 initiatives) 10 8 6 3 %
37%
30%
22%
11%

13. Program Measure 2: Previous Year
Data were reported on 34 initiatives, but most grantees reported insufficient information on this measure. Data are reported below for 12 initiatives (22 initiatives didn’t have enough info).
Met Target
Yes No
Year 2
(2 initiatives) 1
Year 3
(5 initiatives) 4
Year 4
Total
(12 initiatives) 6 %
50%

14. Program Measure 2: Current Year Reporting Problems
DQI was unable to determine if the target was met for three initiatives.
Although the grantees identified measures to assess improvement in implementation, they did not provide enough information for DQI to determine if they were fidelity measures.

15. Program Measure 3: Current Year
Project Costs
Cost of TA
Cost for all PD
% for TA
Year two (22 initiatives)
$7,431,545
$12,258,796
61%
Year three (10 initiatives)
$3,009,936
$3,630,786
83%
Year four (13 initiatives)
$3,570,736
$6,270,716
57%
Total (45 initiatives)
$14,012,217
$22,160,298
63%

16. Program Measure 3: Current Year cont.
Met Target
Data Not available
Unable to Determine
Yes No
Year two
(40 initiatives) 25 2 11
Year three
(13 initiatives) 9 1 3
Year four
(14 initiatives) 4
Total
(67 initiatives) 43 7 6 %
64%
10% 9%
16%

17. Program Measure 3: Reporting Problems
Some grantees reported the percentage of funds used to sustain SPDG-funded practices but did not provide dollar amounts.
DQI could not determine if the target was met for 17 initiatives
data were not reported
grantees with multiple initiatives did not disaggregate by initiative

18. How we need to improve
Getting 3’s and 4’s for more EB-PD components. ---Harder
Continuing to improve how you describe the fidelity measure you are using --Easier
And choosing just one fidelity measure per initiative as a “Program” measure
Using the right equation and numbers for measure Easier
Providing all of the necessary info for each initiative

19. How can we help? Please be specificTA
Networking PD
Feedback
Other?