The Story of a Non-Profit’s Journey to Find the Perfect Evaluation Reporting Tool Kelly Romirowsky, PsyD Evaluation & Research Manager, Jewish Federation of Greater Philadelphia

The Need for Reporting Tools What we evaluate at JFGP and why How data needs to be aggregated How data needs to be presented to donors and funding committees

Why off the shelf tools weren’t good enough Participants cannot save their work and come back - must complete in 1 sitting Limited functionality - no branching (i.e. if you belong to a certain subclass of programs, you answer a particular set of questions) Page time-outs Validation checks Can't control the setup of summary reports (graphically)

Two customized tools Survey Tool- data collection Report Manager Interface (RMI)- reporting Both are excel-based Cost/Number of hours Survey tool- $37,000 10 week period RMI- pulls data automatically from CSV file or survey tool via SQL queries $20,000 6 weeks Polished and professional for quick and easy presentations to funding committees

Survey Reporting Tool 3 center surveys 15 priority area surveys 60 program-specific surveys Surveys hosted like a web page Surveys written in basic HTML from a template SQL database holds survey responses (data) Database is linked to the RMI

Steps Create spreadsheet of questions based on goals and objectives, standards, and program-specific questions Decide which data variables you will want to present in graph/table form Have consultant create survey tool template based on your specifications Go through 2 day training and get manual Create evaluation survey- questions grouped by commonality Create instructional email text inside survey tool (#10)

Steps (cont.) Review survey questions for errors Create final draft of survey and email text Send message to program managers alerting them to upcoming email (to prevent spam) Send out email with link 1 month before due date (#11-13) Field questions from program managers and resolve any bugs (#14-15) Monitor progress of program managers in completing report (#16)

Steps (cont.) Send out final reminder to program managers before due date- own email- to those who haven’t finalized Print out evaluation data for each program after report has been finalized (#17-18) Export data into excel as a CSV file - this is where you would manually change data (#19-20)

RMI steps Open RMI (#21) Adjust RMI reporting dates Re-load database Refresh reports Check various RMI reports for errors Incorporate reports into Powerpoint presentations (#22-24) Report Card

This is the programming language for the instructional e-mail that gets sent out to program managers. It includes a personalized link to the survey.

This is what the actual e-mail looks like.

This is what the end of the e-mail looks like.

This is the start page for the survey.

This is the main menu for a PARTICULAR SURVEY This is the main menu for a PARTICULAR SURVEY. This part of the page shows you the start, finish, and distribution dates of the survey.

This is the main menu for a PARTICULAR SURVEY This is the main menu for a PARTICULAR SURVEY. From here you can resend a survey or add more respondents.

This is the overview report of a survey.

This is what the printout of questions and responses looks like.

This is the printout of questions and responses for the Impact of Evaluation section.

This is the main menu for a PARTICULAR SURVEY This is the main menu for a PARTICULAR SURVEY. From here you can check on the respondents’ progress, export survey response data as a CSV file into Excel, or archive the survey.

This is what the CSV file of data looks like.

This is the RMI main menu.

This is the main menu of the survey tool; this is where you go to create new surveys and access previously created/distributed surveys. This is the main menu of the survey tool; this is where you go to create new surveys and access previously created/distributed surveys.

This is the programming language for the Impact of Evaluation page in the survey.

This is what the Impact of Evaluation page looks like in the survey itself.

This is what happens at the end of a survey if you have not completed all the required questions. Notice how a “FINALIZE” button does not appear. This is what happens at the end of a survey if you have not completed all the required questions. Notice how a “FINALIZE” button does not appear.