1. The Story of a Non-Profit’s Journey to Find the Perfect Evaluation Reporting Tool
Kelly Romirowsky, PsyD
Evaluation & Research Manager,
Jewish Federation of Greater Philadelphia

2. The Need for Reporting Tools
What we evaluate at JFGP and why
How data needs to be aggregated
How data needs to be presented to donors and funding committees

3. Why off the shelf tools weren’t good enough
Participants cannot save their work and come back - must complete in 1 sitting
Limited functionality - no branching (i.e. if you belong to a certain subclass of programs, you answer a particular set of questions)
Page time-outs
Validation checks
Can't control the setup of summary reports (graphically)

4. Two customized tools Survey Tool- data collection
Report Manager Interface (RMI)- reporting
Both are excel-based
Cost/Number of hours
Survey tool- $37,000
10 week period
RMI- pulls data automatically from CSV file or survey tool via SQL queries
$20,000
6 weeks
Polished and professional for quick and easy presentations to funding committees

5. Survey Reporting Tool 3 center surveys 15 priority area surveys
60 program-specific surveys
Surveys hosted like a web page
Surveys written in basic HTML from a template
SQL database holds survey responses (data)
Database is linked to the RMI

6. Steps
Create spreadsheet of questions based on goals and objectives, standards, and program-specific questions
Decide which data variables you will want to present in graph/table form
Have consultant create survey tool template based on your specifications
Go through 2 day training and get manual
Create evaluation survey- questions grouped by commonality
Create instructional text inside survey tool (#10)

7. Steps (cont.) Review survey questions for errors
Create final draft of survey and text
Send message to program managers alerting them to upcoming (to prevent spam)
Send out with link 1 month before due date (#11-13)
Field questions from program managers and resolve any bugs (#14-15)
Monitor progress of program managers in completing report (#16)

8. Steps (cont.)
Send out final reminder to program managers before due date- own - to those who haven’t finalized
Print out evaluation data for each program after report has been finalized (#17-18)
Export data into excel as a CSV file - this is where you would manually change data (#19-20)

9. RMI steps Open RMI (#21) Adjust RMI reporting dates Re-load database
Refresh reports
Check various RMI reports for errors
Incorporate reports into Powerpoint presentations (#22-24)
Report Card

10. This is the programming language for the instructional that gets sent out to program managers. It includes a personalized link to the survey.

11. This is what the actual e-mail looks like.

12. This is what the end of the e-mail looks like.

13. This is the start page for the survey.

14. This is the main menu for a PARTICULAR SURVEY
This is the main menu for a PARTICULAR SURVEY. This part of the page shows you the start, finish, and distribution dates of the survey.

15. This is the main menu for a PARTICULAR SURVEY
This is the main menu for a PARTICULAR SURVEY. From here you can resend a survey or add more respondents.

16. This is the overview report of a survey.

17. This is what the printout of questions and responses looks like.

18. This is the printout of questions and responses for the Impact of Evaluation section.

19. This is the main menu for a PARTICULAR SURVEY
This is the main menu for a PARTICULAR SURVEY. From here you can check on the respondents’ progress, export survey response data as a CSV file into Excel, or archive the survey.

20. This is what the CSV file of data looks like.

21. This is the RMI main menu.

25. This is the main menu of the survey tool; this is where you go to create new surveys and access previously created/distributed surveys.
This is the main menu of the survey tool; this is where you go to create new surveys and access previously created/distributed surveys.

26. This is the programming language for the Impact of Evaluation page in the survey.

27. This is what the Impact of Evaluation page looks like in the survey itself.

28. This is what happens at the end of a survey if you have not completed all the required questions. Notice how a “FINALIZE” button does not appear.
This is what happens at the end of a survey if you have not completed all the required questions. Notice how a “FINALIZE” button does not appear.