Datacenter Health Check Terri-Lynn Thayer AVP/CIO, Computing & Information Services EDUCAUSE Enterprise Technology Conference, May 2007

Copyright Terri-Lynn B. Thayer 2007 This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Presentation Overview Datacenter blood pressure is rising Pre-assessment work Reliability goals Assess existing space, power, cooling, fire suppression, & security Assessment results & options Recommendations for a renovation Key considerations in site selection

Datacenter Blood Pressure is Rising Growing Demands Increased number of servers Increase in breadth of customer base – taking in previously distributed computing Technology Changes Storage growth Power/cooling needs – new boxes are smaller but they are energy hogs and they are hot, hot, hot (10X power for a fully populated rack and 3-4X power to cool it down) Business Resumption Concerns 24 x 7 demands Well publicized disasters - 911, Katrina Many university datacenters today lack standby power generation Research Support National trend for R1s to take on more support centrally Our Datacenters are Old

Pre-Assessment Work Capacity Planning and Growth Analysis Determine a planning horizon Identify services likely to be provided from your datacenter during that time period Business Continuity and Disaster Recovery Objectives How long can your University operate without a functioning datacenter? Do you have a cold or hot site? Should you consider a multiple datacenter approach?

More Homework Research computing support decision is fundamental Review your insurance How much do you have Other requirements of your insurer Get professional help with the assessment Involve your university facilities engineers Seek advice from outside professionals who are familiar with modern datacenter design and operation What cost/risk profile is your institution comfortable with? Reliability goals

BRUNS-PAK Data Center Reliability Ranking BRUNS-PAK 999 NEW DURHAM ROAD EDISON, NJ (732) Fax: (732)

Datacenter Evaluation Space Electrical System Mechanical System Fire Protection System Security

Space Square footage of conditioned space Raised floor Access Elevators Door size Machine room layout Furnishings, racks, command center Is the space expandable? Is this a multi purpose facility?

Electrical/Power Considerations Source and costs Patch Panel/Power Control Units/ Power Distribution Units Standby power Uninterruptible Power Supply (UPS) Redundant/non-redundant Battery type (wet vs dry), capacity, and monitoring Generator Type Power and cooling Are the systems expandable? Delicate balance Other Surge protection, lightning protection, grounding

Mechanical Systems Evaluation Cooling & humidity control Chilled water – do you have a dedicated chiller? Computer Room Air Conditioner - CRAC units - # and location Capacity and reliability Heat Detectors Airflow distribution Water sensors

Fire Protection Systems Detection Smoke alarms Heat detectors Air sampling Abatement Halon system (production banned in 1994) Full flooding clean agent system FM-200, NAF, Inergen etc Sprinkler system Wet Pre-action

Security Physical access to the facility elevators & doors Caged areas and visitors Multi-purpose facility Door access system Windows Monitoring Closed circuit TV

Ancillary Services and Support Ancillary Services Tape storage Secure storage/staging Paper storage Test/setup lab Printers and print support General storage room Break room Other staffing and services which are provided from your datacenter Machine hosting and associated SLAs others University departments groups external to the University

Assessment Results & Options Most of us will find that our datacenters are not adequate for the anticipated growth over the next five years Majority will identify power and cooling as the most significant issue Cooling and the power to cool will be the number one issue Space constraints will be the runner up

Options Renovate Build a new datacenter Both of the above Multi-datacenter campus Outsource or Hosting

Recommendations for Renovation Implement standby power generation capable of supporting both power and cooling Remove ancillary services from machine room and relocate to other spaces Trade off between space and density is a complex issue High density racking results in significant heat and power provision issues It is general cheaper to provide more space than to keep a small space with high density equipment adequately powered and cooled

Improve Air Flow & Circulation Provide additional space between racks to promote air circulation Open up plenum space by relocating cabling to overhead trays Increase height of the raised floor if possible Consider new cooling solutions and rack technologies – everything old is new again Chilled water is far more efficient than cool air for heat removal Reconfigure the layout to implement a double hot aisle/cold aisle configuration Distribute high density racks

Double Hot Aisle/Cold Aisle hot aisle/cold aisle layout is where cold air is segregated in front of equipment cabinets and hot exhaust air is expelled behind equipment cabinets. This layout eliminates the direct transfer of hot exhaust air from one system into the intake air of another system Double A CRAC unit is located between two hot aisles

Site Selection for a New Facility An opportunity to consider cost Corporate world has moved their datacenters in some cases quite remote from the rest of their operation which allows them to consider Power costs Real estate cost Labor costs

Look for a Old Supermarket Single story Single use facility Slab Few windows Lots of open space around the building Loading docks and delivery truck access

Other Considerations Voice and data connectivity Flooding and other weather related issues If moving to a multi-datacenter approach as part of a business continuity plan then consideration should be given to put the two datacenters at a sufficient distance to reduce dependence on the same power grid and to minimize weather and other regional disasters

Staffing Implications Data Center Managers will need to be more skilled in the area of environmental issues, engineering, and server technologies Facilities organizations may need to devote more time and specialization to cooling and power technologies related to the datacenter Managing data center renovation or build projects will be resource intensive and may result in downtime for key services

Go Green Reduce energy costs (datacenter build may be more expensive) Legislation Environmental concerns and institutional plans to reduce carbon emissions Vendor products Rack and server cooling technologies CO2 for cooling, DC power systems Design Considerations Solar panels and wind energy Heat recycling

Conclusion Our datacenters are under significant stress If we take a close look we will find that most of us will experience power and cooling problems in the near future. Cooling and the power to cool will be the most substantial issue we face. There are new technologies and best practices which will provide some relief Many of us will build new datacenters over the next five years and we should consider remote locations, outsourcing, and green IT solutions These project will require both significant financial resources as well as IT and Facilities staff time. We may need to employ new skill sets It is highly recommended that you engage professional assistance to evaluate your facility and to assist in renovation and new build designs

Additional Resources Past Meetings Spring 2006 meeting Data Center Futures Workshop Presentations