1. Evolution of Program Review at UCF Using SAS Business Intelligence Tools to move Towards Optimization Patricia Ramsey & Carlos Piemonti 2013 FAIR Conference January 31, 2013

2. Task: Examine the quality and productivity of programs - Mandated by state statute - 7 year cycle - Approximately 30 programs per year Goal is to determine which programs to: Enhance, Reduce, Maintain, Eliminate, or Study Further Process requires a self-study, external consultant visit, and committee recommendations to the Provost, BOT & BOG Program Review Process

3. Program Review Data Value Wide range of data is required to support the process during all steps  Departments and programs - self-study prep  External Consultants - better understand the program prior to site visit  Program Review Committee members and the Provost - to identify opportunities or weaknesses Program Review Data Value

4. Each program review data document includes the following: Series of data points Trend information Comparison of program to overall department and college Program Review Data Requirements

5. CycleData Generation Data OutputData Delivery 2000ManualMS Word TablesEmailed 2003ProgrammaticMS Access summary tables Web display via.asp 2006ProgrammaticSAS® Enterprise Guide (EG) to MS Excel pivot tables Emailed & Web (.pdf) 2008ProgrammaticEG – developed report format to.pdf Web (.pdf) 2012ProgrammaticEG and WRS – report surfaced on web Web Program Review History

6. 2006 SAS EG 2012 SAS BI Progress Through the Years 2000 MS Word

7. 2007

8. 2011

9. 2012

10. Recent Advances Surface the data on the web via SAS BI tools – Web Report Studio (WRS) (IR waited for cascading parameters to be available!) Provide data for all programs, not just programs under review Auto-populate the self-study templates Graphs and charts are now developed in Web Report Studio not in EG

11. Old Way

12. New Way Update filters to reflect new reporting cycle

13. The Magic Behind the Scenes Official data is loaded to Data Warehouse each term SAS Information Maps create a view of the data designed for a purpose SAS Web Report Studio graphically surfaces the data on the web

14. New Maintenance Routine 1.Data prep: run an EG project once a year and Roll forward one cohort in each report by updating filters (Feb) 2.Request refreshed data to UAPS (Projections – May), UAPS does it by replacing table 3.Generate reports: preliminary (Apr); final (Aug) for programs under review

15. New Program Review Cycle Preparation Update ‘Year’ filters for next cycle – in each tab in each report SAS Web Report Studio Feb Update source data to include for next reporting year – Automatically Info Maps are refreshed SAS Enterprise Guide Feb University Community has access now! 1H

16. New Program Review Cycle PDF Report Generation Respond to prompts and print reports as PDFs (PMP) Edit PDFs manually Combine PDFs into 1 report per ‘program area’ and update bookmarks Days 1-2 Day 3 Day 4 Program Review 2012-2013: 12 department reports were created twice (preliminary and final versions) in less than 10 workdays.

17. Pegasus Mine Portal https://my.ucf.edu/

18. College Department Undergraduate Graduate Report Contents Sample

19. Prompts Page on the Portal Dynamically generate values & Cascading parameters

20. Cascading Parameters

21. Prompts Sample

22. Report Display in the Portal

23. Update Filters

24. External Access web

25. Program Review 2012-2013 by Department/Program Area Communication Dean's Office UGST Health Mgmt. & Informatics FL Interactive Entertainment Academy Modern Languages Legal Studies Physical Therapy Public Administration Optics Political Science Office of Interdisciplinary Studies Health Professions Document Layout Key 1 12 For a total of 26 programs!

26. Limitations  Security - Program Review in the portal is limited to select users. Consultants must have access to data outside the portal. IR must make.pdf versions and post to a location outside this firewall so they have easy access  Design - SAS has not yet designed prompts that apply to the entire report. The user must respond to prompts on each tab