1 Monitoring & Evaluation Plan Bea iSTEP 2009

2 Monitoring – Why? Are we on the right track? Do we need corrective action? What are the problems? Are we wasting resources/time on something? What are we missing?

3 Monitoring – What? Activity/ProjectWhat to monitor Needs AssessmentEnd user reactions / feedback Cell Phone Project Technology progress & compatibility with needs Literacy Project Technology progress & compatibility with needs Braille Tutor Project Technology progress & compatibility with needs Documentation / Dissemination Updates to blogs, journals and other media iSTEP Student Learning Lessons learned so far – introspection & group discussion

4 Monitoring – How? Weekly group “huddles” Plan vs. Actual comparison Feedback from partner (Eric Beda) – possibly weekly End user interactions – as many as possible Reality checks

5 Evaluation – Why? My thesis depends on it Understand whether we are making an impact Measure significance of impact Identify areas for improvement Produce transparent results to stakeholders Make necessary adjustments

6 Evaluation – When? Phase 1: Pre-Intervention Measurements – What is the current status? Phase 2: Intermediate Assessment – Are we making progress toward our goals? Phase 3: Final Evaluation – Were we successful? Final Phase: iSTEP Overall Assessment – Did iSTEP students gain from the experience?

7 Evaluation – What? Activity/ProjectMetric(s) Cell Phone Project Number of new entries into database, before, during and after intervention Literacy Project Student test scores before, during and after intervention: tests may include spelling, reading and writing Braille Tutor Project Student test scores before, during and after intervention: tests may include writing and reading in Braille language Campaign Number of comments on FB page, Hits on website, Amount of blog readership, Twitter activity, etc.

8 Evaluation – How? Collect data! Questionnaires/Surveys to end users Field testing of technology Continuous interviews/conversations with partners and end users Determine, discover, and redefine metrics as necessary

9 Potential Difficulties Data may be scarce and difficult to obtain Evaluation settings will not be ideal – Lots of confounding factors! – Statistical testing might be difficult Participation in surveys and questionnaires might be low Language and/or cultural barriers Time management will be tough – setting things up will be time-consuming May not obtain accurate responses from end-users Traveling to and from project locations could be costly – Might limit interactions with end-users Unexpected delays and issues may throw off evaluation

10 Questions? Comments? Thank you for listening Suggestions Welcome and Appreciated!