Who Hit The Turbo Button? February 13, 2013 Presenter: Stephen Watson ITS System Manager Stephen F. Austin State University
Stephen Watson Systems Manager ITS Previous jobs at SFASU (Programmer Analyst in Student/HR/Budget) (Systems Programmer I and II) 18 Years at SFASU Gabriel Nguyen Sr. Sales Consultant Oracle Corporation
Agenda How SFA went from sluggish to turbo performance... Introduction Challenges Architecture Selection Process Exadata Implementation Process Performance Results Where We Are Today Q and A
Stephen F. Austin State University Academic and Research University Undergraduate, Graduate and Doctoral Enrollment of 13,000 Both Traditional and Online Courses
SFA Support Staff 3 DBA’s 7 Systems 11 Programming
Current Environment Ellucian Banner 8 Student Human Resources Finance Financial Aid WebFOCUS Reporting UC4 AppManager Oracle 11g R2 DB and 11gR2 WLS/FMW
Why Did We Need A New DB Platform? Registration Performance Issues Future DB Platform Support from Oracle and Ellucian
Problem 1: Registration Performance 2008 Banner 8 Go-Live Registration load issues during orientations and pre-registration periods Limited to simultaneous users Had to break registration in very small groups (<500) Student Experience Impact
Problem 1: Registration Performance University reputation (bad press, social media…) Student satisfaction declined Negatively impacted enrollment Negative comments from parents about experience Peripheral Impact
Making the News
Problem 2: Product End of Life HP de-support Open VMS Ellucian de-support of Open VMS Oracle de-support of Itanium platform HPUX/11G would be de-supported soon soon as well Technology Impact
Environment Before Exadata HP Itanium rx6600 Running Open VMS 8.3 and Oracle 10G R2 (3 identical servers) Banner Production on one server Banner Test instances (4) on second server Banner Prod DR on third server, ready to come up when needed No high-availability, data protection, or disaster recovery capabilities
Hardware Refresh Time We could run HPUX/11G on the servers but this would be de-supported soon as well After getting another year from the hardware we would have to buy new hardware
Why we selected Exadata Investigation and research Oracle EX-CITE conducted on campus Key advantages Fast transaction processing Extreme performance and redundant appliance type architecture One throat to choke (so to speak) One process to patch everything (OS, switches, storage and database homes) Lots of storage capacity and processing power for future growth
What is Exadata? Oracle’s Engineered Database Machine Pre-configured, pre-integrated and highly efficient combination of hardware and software to provide a complete solution to the customer Optimized for OLTP and DW database workloads The Exadata Quarter “Server” or Rack contains Multiple database nodes (2) Multiple Cell or storage servers (3) Shared storage (12 3 TB disk per cell ) All connected with a fast Infinband network (40 GB/sec) Designed to work as a unit to increase transaction performance Lots of RAID redundant storage. (108 TB raw ) ASR Server monitors hardware failures continually
Solution: SFA Exadata Platform X2-2 Quarter Rack 2 Sun Fire x4170 DB Servers 3 Exadata Storage Servers (SAS) Raw Disk – 108TB (SAS Drives) Raw Flash – 1.2TB User Data – 40TB ( No Compression) ZFS Backup Area – 20 TB Prod X2-2 Quarter Rack DR/Dev/Test/QA X2-2 Quarter Rack Active Data Guard (physical standby) 10g to 11g migration RAC 3 databases, 6 Instances 700 GB size Stand-by PROD databases 5 databases, 10 Instances 3.0 TB size OEM Grid Control Grid Control Console GigE Backup Network ZFS Storage Appliance Backups of database Export dumps Data Center A – Site 1Data Center B – Site 2
Bottom Line SFA needed a solution that “just works” SFA enrollment growing – need a solution that will keep up with demands of administration and students Opportunity for IT consolidation to lower costs Avoid further loss of confidence from faculty/staff in our IT management and staff
Bottom Line Loss of confidence from the students in SFA being able to deliver on their mission promise: “...comprehensive institution dedicated to excellence in teaching, research, scholarship, creative work, and service” To effectively recruit quality students in the future and confidence from parents in SFA being the right university for their family members
Project Timeline A very rapid deployment DB Export Started on 4/5/2012 at 5:00 pm DB import into Exadata Completed 4/7/2012 Testing and Data Validation 4/7 thru 4/8/2012 Go live Sunday 4/8/2012 at 7:00 pm
Summary of Implementation Delivery to go live one of the shortest I have ever been involved with Lots of long hours including evenings and weekends (several hour weeks) Oracle could not believe we implemented in this short time frame Usual implementation of Exadata (6 months to 1 year) Ellucian and Oracle consulting were crucial
Where we are today 2 successful large registrations, fall and spring 5 successful orientations (registration) Returned to larger groups for registration Classifications (about 3000 per group) All Processes running faster Better faculty/staff experience in Banner Better student experience with registration
Where we are today (continued) Administration very pleased with the success of the registration process Strategically positioned for future IT consolidation of databases on campus No complaints from students
Benefits Time to Value Return on Investment Performance Before: Times After: Times Performance Improvement Query run time Reporting 1 to 3 minutes10 to 30 seconds6 fold increase AR Processes Other Processes 5 Min to 3 Hours5 Sec to 10 min60 fold increase Backup TimesFull Hot Backups 4 Hours Full Hot Backups 2 hours Time cut in half RegistrationLess than 200 concurrent Up to 1000 concurrent 5 Fold increase
Highlights One Vendor for OS and DB issues Oracle patches everything (OS, DB, Switches and Storage) More disk space than we know what to do with FLASH CACHE storage on disk helps performance
Highlights Lots of room for future growth No system tweaking - delivered for performance Faster backups Faster cloning
Questions? For additional information contact: Stephen Watson IT System Manager Information Technology Services