1. Solving the OSP Reporting Conundrum Lynn Ward, Principal Systems Analyst Indiana University

2. Indiana University 7 Campuses 2 large research campuses (Bloomington and IUPUI) 5 smaller regional campuses (East, Fort Wayne, Kokomo, Northwest, South Bend, Southeast) Stats > 1K degree programs > 5K FT, 2K PT Faculty > 11K staff > 100K students Sakai is our only CMS/LMS July 200910th Sakai Conference - Boston, MA, U.S.A.2

3. ePortfolio Timeline @ IU 1998: IUPUI campus adopts a competency or ability-based approach to general education 2000: IUPUI campus initiates Eport project, partnering with Ali Jafari (Epsilen) 2003: IU and rSmart partnership on OSP 2.0 Mellon Grant; IUPUI abandons first homegrown tool 2004-05: early pilots with OSP 2006: moratorium on new portfolio projects 2007-present: growing interest and use on all campuses July 200910th Sakai Conference - Boston, MA, U.S.A.3

4. OSP Use @IU > 20 active portfolio projects underway, many in professional schools Majority focus on outcomes assessment at the program level Highly individualized approach to assessment Individual differences accommodated via customized scaffoldings and forms July 200910th Sakai Conference - Boston, MA, U.S.A.4

5. OSP Support @ IU.5 – 2 FTE developers.10-20 FTE designer.25 FTE combined consultant, analyst, tech writer, trainer, QA specialist Outsource some QA and development work Growing knowledge among consultants in campus teaching centers July 200910th Sakai Conference - Boston, MA, U.S.A.5

6. The Conundrum Project coordinators need to aggregate, analyze, and summarize evaluation (and other types of) data Evaluation forms differ widely in terms of # of rating fields # rating scale # labels assigned to each point on scale XML Programmer required to develop custom reports for each project – DOES NOT SCALE July 200910th Sakai Conference - Boston, MA, U.S.A.6

7. Our Solution Define local form standards Flexible Predictable Publish and promote standards Define report definitions based on those standards Reports “discover” the XSD structure on the fly and render the results July 200910th Sakai Conference - Boston, MA, U.S.A.7

8. Evaluation Form Standard Field name conventions: Rating fields (rating, rating 1, rating2, rating3 …rating20) Comments (comments) Rating fields must be represented as a selection field (drop-down menu) All rating fields in the form (except rating) must use the same rating scale. Enumeration values must be a number, “n/a”, or null Rating fields may not be repeatable July 200910th Sakai Conference - Boston, MA, U.S.A.8

9. Customization Options Rating fields and comments fields may use any display name Rating scales may have any number of points The display label for each point on the scale may be any valid string Evaluation forms may have any valid name Each matrix cell may use a different evaluation form. July 200910th Sakai Conference - Boston, MA, U.S.A.9

10. SAMPLE REPORTS July 200910th Sakai Conference - Boston, MA, U.S.A.10

11. 10th Sakai Conference - Boston, MA, U.S.A.11 Rating Summary

12. Rating Details July 200910th Sakai Conference - Boston, MA, U.S.A.12

13. Interactive Report – Main (Matrix) View July 200910th Sakai Conference - Boston, MA, U.S.A.13 Click to see summary report for cell

14. Interactive Report – Summary View July 200910th Sakai Conference - Boston, MA, U.S.A.14 Click to see detailed report for cell Click to return to main (matrix) view.

15. Interactive Report – Details View July 200910th Sakai Conference - Boston, MA, U.S.A.15 Click to return to summary view Click to return to main (matrix) view. Click to open cell in new window.

16. Matrix Cell Status Summary July 200910th Sakai Conference - Boston, MA, U.S.A.16

17. Matrix Cell Status Details July 200910th Sakai Conference - Boston, MA, U.S.A.17

18. Matrix Link Summary July 200910th Sakai Conference - Boston, MA, U.S.A.18

19. Next Steps Publish form templates and documentation on how to customize Refine and augment existing reports based on user feedback. Merge Reports tool enhancements to trunk Post report definitions to OSP library so other institutions can use them. July 200910th Sakai Conference - Boston, MA, U.S.A.19

20. For More Information Contact… – Lynn Ward (leward@iupui.edu)leward@iupui.edu – Chris Maurer (chmaurer@iupui.edu)chmaurer@iupui.edu – Bryan Holladay (bahollad@iupui.edu)bahollad@iupui.edu July 200910th Sakai Conference - Boston, MA, U.S.A.20