Evolution of Program Review at UCF Using SAS Business Intelligence Tools to move Towards Optimization Patricia Ramsey & Carlos Piemonti 2013 FAIR Conference January 31, 2013

Task: Examine the quality and productivity of programs - Mandated by state statute - 7 year cycle - Approximately 30 programs per year Goal is to determine which programs to: Enhance, Reduce, Maintain, Eliminate, or Study Further Process requires a self-study, external consultant visit, and committee recommendations to the Provost, BOT & BOG Program Review Process

Program Review Data Value Wide range of data is required to support the process during all steps  Departments and programs - self-study prep  External Consultants - better understand the program prior to site visit  Program Review Committee members and the Provost - to identify opportunities or weaknesses Program Review Data Value

Each program review data document includes the following: Series of data points Trend information Comparison of program to overall department and college Program Review Data Requirements

CycleData Generation Data OutputData Delivery 2000ManualMS Word Tables ed 2003ProgrammaticMS Access summary tables Web display via.asp 2006ProgrammaticSAS® Enterprise Guide (EG) to MS Excel pivot tables ed & Web (.pdf) 2008ProgrammaticEG – developed report format to.pdf Web (.pdf) 2012ProgrammaticEG and WRS – report surfaced on web Web Program Review History

2006 SAS EG 2012 SAS BI Progress Through the Years 2000 MS Word

2007

2011

2012

Recent Advances Surface the data on the web via SAS BI tools – Web Report Studio (WRS) (IR waited for cascading parameters to be available!) Provide data for all programs, not just programs under review Auto-populate the self-study templates Graphs and charts are now developed in Web Report Studio not in EG

Old Way

New Way Update filters to reflect new reporting cycle

The Magic Behind the Scenes Official data is loaded to Data Warehouse each term SAS Information Maps create a view of the data designed for a purpose SAS Web Report Studio graphically surfaces the data on the web

New Maintenance Routine 1.Data prep: run an EG project once a year and Roll forward one cohort in each report by updating filters (Feb) 2.Request refreshed data to UAPS (Projections – May), UAPS does it by replacing table 3.Generate reports: preliminary (Apr); final (Aug) for programs under review

New Program Review Cycle Preparation Update ‘Year’ filters for next cycle – in each tab in each report SAS Web Report Studio Feb Update source data to include for next reporting year – Automatically Info Maps are refreshed SAS Enterprise Guide Feb University Community has access now! 1H

New Program Review Cycle PDF Report Generation Respond to prompts and print reports as PDFs (PMP) Edit PDFs manually Combine PDFs into 1 report per ‘program area’ and update bookmarks Days 1-2 Day 3 Day 4 Program Review : 12 department reports were created twice (preliminary and final versions) in less than 10 workdays.

Pegasus Mine Portal

College Department Undergraduate Graduate Report Contents Sample

Prompts Page on the Portal Dynamically generate values & Cascading parameters

Cascading Parameters

Prompts Sample

Report Display in the Portal

Update Filters

External Access web

Program Review by Department/Program Area Communication Dean's Office UGST Health Mgmt. & Informatics FL Interactive Entertainment Academy Modern Languages Legal Studies Physical Therapy Public Administration Optics Political Science Office of Interdisciplinary Studies Health Professions Document Layout Key 1 12 For a total of 26 programs!

Limitations  Security - Program Review in the portal is limited to select users. Consultants must have access to data outside the portal. IR must make.pdf versions and post to a location outside this firewall so they have easy access  Design - SAS has not yet designed prompts that apply to the entire report. The user must respond to prompts on each tab