Ph.D. Proposal Presentation TRANSIENT REDUCED-ORDER CONVECTIVE HEAT TRANSFER MODELING FOR A DATA CENTER Rajat Ghosh G.W. Woodruff School of Mechanical Engineering Georgia Institute of Technology Atlanta, GA 30332-0405 September 25, 2012

Ph.D. Proposal Presentation Outline Introduction Problem Statement Representative Case Studies Case study-1 Case Study-2 Case Study-3 Remaining Deliverable Dissertation Timeline Closure Introduction Ph.D. Proposal Presentation

Increasing energy consumption UB: Upper Bound LB: Lower Bound (Based on data reported by J. Koomey in the New York Times, July 31, 2012.) Need to improve energy efficiency in data centers (DCs). Introduction Ph.D. Proposal Presentation

Increasing power density http://download.intel.com/technology/eep/data-center-efficiency/state-of-date-center-cooling.pdf Datacom Equipment Power Trends and Cooling Applications (2005), ASHARE TC 9.9 Frequency of occurance Need of high-resolution monitoring and feedback control Both in temporal and spatial dimensions. Introduction Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Dynamic data centers (Liu, J.,Terzis, A., "Sensing data centers for energy efficiency,“Phil. Trans. R. Soc. A (2012)) (CRAC supply temperature data from CEETHERM: Sept. 9-10, 2012, 11 pm -11pm ) Rapidly changing server load leads to dynamic thermal environment -Dynamic thermal analysis requires fast (near-real-time) modeling algorithm. -State-of-the-art CFD/HT frameworks are too sluggish. Need of fast surrogate modeling algorithm Introduction Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Data center cooling 1/3 of energy spent in a DC is dedicated to its cooling systems. Forced convective air cooling: - Heat generated at chips dissipates via cooling airflow propelled by fans in the computer room air conditioning (CRAC) units. http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/white_paper_c11-627731_ps10280_Products_White_Paper.html Various airflow schemes exist: - Underfloor plenum supply and ceiling return airflow scheme. Introduction Ph.D. Proposal Presentation

Multiscale thermal system Turbulent Convection Turbulent Convection Turbulent Convection + Conduction Conduction Involvement of several decades of length and time scales Spatial: 5 decades (mm to Dm). Temporal: 4 decades (10-2 s to 10 s). Introduction Ph.D. Proposal Presentation

Current Approaches for transient thermal modeling Model Accuracy Lumped System Modeling Computational fluid dynamics/ Heat Transfer (CFD/ HT) Modeling Reduced-order Modeling Involves iterative solution of non-linear conservation equations. Involves posing zero local gradient condition. Optimal and controllable Trade-off. Computational Time Introduction Ph.D. Proposal Presentation

Types of Reduced-order Model (ROM) Statistical Response Surface Model Modal Reduction-based Low- Dimensional Model Simplified Physics-based Model Proper Orthogonal Decomposition (POD/ PCA) Nonlinear Volterra Theory Laplacian Model Thermal Zone Model Harmonic Balance Approximation Introduction Ph.D. Proposal Presentation

computation for a Transient CFD Simulation DC Modeling Requirement m spatial nodes and n time steps. Restriction on temporal discretization: The dependent variables for the turbulent convective temperature field: u, v, w, T, ε, k. Computational step~ O(n(m3 + 4m)) m3: For solving momentum equations together. 4m: For solving pressure correction (continuity)+Temperature + Turbulence n: Number of time steps For a rack-level simulation: m~ 1.4 millions, n~1 t~2 hours in a 5.6 GHz machine Introduction Ph.D. Proposal Presentation

Computation for a Reduced order modeling DC Modeling Requirement: m dimensional temperature field with n transient observations. Initial data is collected via measurements or CFD. POD/Interpolation-based reduced-order modeling Computational step~ O(3mn+log(n)+kn+n2+k2)) 3mn: Row-wise average + Generation of parameter-dependent component+ Generation of covariance matrix log(n): Proper orthogonal decomposition of covariance matrix (Power algorithm). kn: Finding POD coefficients for the input parameter space n2: Interpolation k2: Computation of new data (k=principal component number). No higher power of m. Introduction Ph.D. Proposal Presentation

Limitation of Existing Modeling Algorithm Computational fluid dynamic and heat transfer (CFD/HT) modeling Too sluggish to be fit for a near-real-time modeling algorithm. Stochastic nature does not warrant expensive CFD simulations. Reduced-order modeling A few studies exist with time as the parameter. No study exists with spatial location as the parameter. No multi-parameter model exists. Few studies use experimental data as model input: use of CFD defeats the purpose of using ROM. Need an alternative to Galerkine projection-based POD coefficient determination. Problem Statement Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Scope of dissertation Development of measurement-based parametric modeling framework One parameter model For improving temporal resolution. For improving spatial resolution. Multi-parameter model For improving resolution in an additional dimension like rack heat load. Development of interconnected multiscale model - Hybrid reduced-order modeling approach. Problem Statement Ph.D. Proposal Presentation

single parameter (Time) Reduced-order Algorithm Proper Orthogonal Decomposition (POD)-based modal reduction. Time is the modeling parameter. Reduces the sampling rate Use interpolation/ extrapolation to determine POD coefficients Avoid computationally-prohibitive Galerkin projection. Representative Case Study Ph.D. Proposal Presentation

Case study for single parameter (TIME) POD Model After remaining shut down for 2 minutes, the CRAC unit is turned on at t=0. Representative Case Study Ph.D. Proposal Presentation

Optimality of POD Modes First 10 POD modes capture more than 90% characteristics of the temperature field. Representative Case Study Ph.D. Proposal Presentation

Principal Component Number As captured energy percentage increases, the corresponding principal component number increases. Representative Case Study Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Error Formulation Representative Case Study Ph.D. Proposal Presentation

Temperature measurement Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire. Response time 20 ms. Measurement Frequency: 1 Hz. x-axis: Parallel to rack width. y-axis: Parallel to tiles. z-axis: parallel to rack height. S. Ravindran, Error Estimates for Reduced Order POD Models of Navier-Stokes Equations, ASME IMECE, 2008, pp. 652-657. Representative Case Study Ph.D. Proposal Presentation

POD/ Interpolation framework Temperature map at the rack inlet at t=92 s. A posterior measurement, t~100 s An extra step of interpolation, t~10 s Deviation~ O(1%) POD model is efficient in improving parametric resolution of transient temperature data Accuracy of POD model prediction is identical to experimental data. Representative Case Study Ph.D. Proposal Presentation

Calibration of analytical error Calibrated analytical error obviates the necessity of determining a posteriori prediction error . Representative Case Study Ph.D. Proposal Presentation

POD/ Extrapolation framework Temperature map at the rack inlet at t=207 s. A posterior measurement, t~207 s An extra step of interpolation, t~10 s Deviation~ O(5%) POD model is efficient in improving parametric resolution of transient temperature data Accuracy of POD model prediction is identical to experimental data. Representative Case Study Ph.D. Proposal Presentation

Calibration of analytical Error Calibrated analytical error obviates the necessity of determining a posteriori prediction error . Representative Case Study Ph.D. Proposal Presentation

single parameter (SPACE) Reduced-order Algorithm Proper Orthogonal Decomposition (POD)-based modal reduction. Coordinates of spatial location are the modeling parameters. Improves the granularity of experimental data. Reduction in sensor density. Representative Case Study Ph.D. Proposal Presentation

Case study for single parameter (Space) POD Model Sudden shut down of the CRAC unit and power back after 100 s. (Photo courtesy to IBM) Representative Case Study Ph.D. Proposal Presentation

Prediction for DOF-1 Points Improves spatial resolution between (70, 51, -1) and (70, 50, -1). Representative Case Study Ph.D. Proposal Presentation

Prediction for Dof-2 points Improves spatial resolution between (56, 31, 2.5) and (56,30,5.5). Representative Case Study Ph.D. Proposal Presentation

TWO parameter Reduced-order Algorithm POD-based modal decomposition. Time and rack heat load as the modeling parameters. Representative Case Study Ph.D. Proposal Presentation

Case study for multi-parameter (Time, rack heat load) POD Model Sudden shut down of the CRAC unit and power back after 100 s (t=0 in the plot). Representative Case Study Ph.D. Proposal Presentation

Comparison for Extrapolation at Q=1500 W Extrapolation in time and interpolation in heat load. Representative Case Study Ph.D. Proposal Presentation

Interconnected Multiscale modeling Experimentally validated CFD/HT modeling for a selected part of the CEETHERM data center laboratory. Development of the hybrid modeling framework combining finite network modeling (FNM) and POD for simulating a selected part of the CEETHERM data center laboratory. Comparison and validation. Deliverable Ph.D. Proposal Presentation

Dissertation TimeLine May 2010-Dec. 2011 Single-parameter POD framework development with time as the parameter. Jan. 2012-June 2012 Single-parameter POD framework development with spatial location(s) as the parameter(s). June 2012-Oct. 2012 Multi-parameter POD framework development with spatial location and time as the parameters. Nov. 2012- April 2013 Interconnected multi-scale modeling. Planning Ph.D. Proposal Presentation

Ph.D. Proposal Presentation publications REFEREED JOURNAL PUBLICATION Rajat Ghosh, and Yogendra Joshi, “Error Estimate in POD-based Dynamic Reduced-order Thermal Modeling of Data Centers,” International Journal of Heat and Mass Transfer (Revised version submitted). REFEREED CONFERENCE PUBLICATIONS Rajat Ghosh, Levente Klein, Yogendra Joshi, and Hendrik Hamann, “Reduced-order Modeling Framework for Improving Spatial Resolution of the Temperature Data Measured in an Air-cool Data Center,” Semi-Therm, San Jose, California, March 17-21, 2013. Rajat Ghosh, Vikneshan Sundaralingam, and Yogendra Joshi, “Effect of Rack Server Population on Temperatures in Data Centers,” Intersociety Thermal Conference (ITherm), San Diego, California, May 30-June 1, 2012. Rajat Ghosh, Vikneshan Sundaralingam, Steven Isaacs, Pramod Kumar and Yogendra Joshi, “Transient Air Temperature Measurements in a Data Center,” Indian Society of Heat and Mass Transfer Conference, Chennai, India, Dec. 27-30, 2011. Rajat Ghosh, Pramod Kumar, Vikneshan Sundaralingam, and Yogendra Joshi, “Experimental Characterization of Transient Temperature Evolution in a Data Center Facility,” International Symposium on Transport Phenomena, Delft, the Netherlands, Nov. 8-11, 2011. Rajat Ghosh and Yogendra Joshi, “Dynamic Reduced-order Thermal Modeling of Data Center Air Temperatures,” InterPACK, Portland, Oregon, July 6-8, 2011. Planned PUBLICATIONS Closure Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Acknowledgement The support for this work from IBM Corporation, with Dr. Hendrik Hamann as the Technical Monitor, is acknowledged . Acknowledgements are also due to the United States Department of Energy as the source of primary funds. Additional support from the National Science Foundation award CRI 0958514 enabled the acquisition of some of the test equipment utilized. The support from the G.W. Woodruff School of Mechanical Engineering as a Graduate Teaching Assistant is acknowledged. The collaboration, goodwill, and help received from all CEETHERM and METTL members (particularly Vikneshan Sundaralingam, Vaibhav Arghode, Pramod Kumar, Steven Isaacs) are highly appreciated. Closure Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Thank YOU! closure Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Appendix Introduction Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Introduction Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Introduction Ph.D. Proposal Presentation

Ph.D. Proposal Presentation Introduction Impact of proliferated cloud computing-based e-commerce services on data Centers: Increasing dynamic characteristics. Increasing energy consumption. Increasing power densities of racks. Effect on cooling 30%-40% energy consumed by cooling systems. Importance of local thermal characteristics. Need High resolution (space/time) temperature monitoring. Near-real-time feedback control for temperature. Introduction Ph.D. Proposal Presentation

Temperature measurement Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire. Response time 20 ms. Measurement Frequency: 1 Hz. x-axis: Parallel to rack width. y-axis: Parallel to tiles. z-axis: parallel to rack height. Introduction Ph.D. Proposal Presentation

Dissertation time line Development of a single-parameter POD-based framework for transient convective heat transfer modeling for an air-cool data center (May 2010-Dec. 2011). Development of a grid-based thermocouple network for transient air temperature measurements (Nov. 2010-Nov. 2011). Development of design protocol for filling out an empty rack (Nov. 2011-Dec. 2012). Development of a single-parameter POD-based framework capable of improving spatial resolution of transient temperature data (Mar. 2012-June 2012). Development of a two-parameter POD-based framework for transient convective heat transfer modeling for an air-cool data center (May 2012-Dec. 2012). Development of a scale-linking across various length-scales in a data center (Oct. 2012-Mar. 2013). Ph.D. dissertation defense (Mar. 2013). Introduction Ph.D. Proposal Presentation

Ph.D. Proposal Presentation May 2010-Dec. 2010 Jan. 2011-June 2011 July 2011-Dec. 2011 Jan. 2012-June 2012 June 2012-Dec. 2012 Jan. 2013-May 2013 Single-parameter POD Framework Development Introduction Ph.D. Proposal Presentation

Literature REVIEW Paper/ Thesis Comments S. V. Patankar, Airflow and Cooling in a Data Center, Journal of Heat Transfer 132 (2010) 73001-1-73001-17. CFD/HT simulation for a steady data center. A.H. Beitelmal, C.D. Patel, Thermo-Fluids Provisioning of a High Performance High Density Data Center, Distributed and Parallel Databases, 21, 227–238, 2007. CFD/HT simulation for a transient data center. Shawn Shields, Dynamic Thermal Response of the Data Center to Cooling Loss during Facility Power Failure, Masters Thesis, Georgia Tech, 2009. Measurement-based transient characterization of a data center. J. D. Rambo, Reduced-order Modeling of Multiscale Turbulent Convection: Application to Data Center Thermal Management, Ph.D. Dissertation, Georgia Tech, 2007. Reduced-order modeling of data center and a posterior error analysis. Introduction Ph.D. Proposal Presentation

Literature REVIEW Contd. Paper/ Thesis Comments Q. Nie, Experimentally Validated Multiscale Thermal Modeling of Electronic Cabinets , Ph.D. Dissertation, Georgia Tech, 2008. Interconnected multiscale modeling for a rack. Graham Nelson, Development of an Experimentally-Validated Compact Model of a Server Rack, Masters Thesis, Georgia Tech, 2009. Development of grid-based temperature measurement system and compact modeling of a server. E. Samadiani, Energy Efficient Thermal Management of Data Centers via Open Multi-scale Design, Ph.D. Dissertation, Georgia Tech, 2009. Reduced-order open design for a multi-scale data center. V. López and H. F. Hamann, Heat transfer modeling in data centers, International journal of heat and mass transfer 54 (2011) 5306-5318. Simplified Physics-based Laplacian Model for a data center. Introduction Ph.D. Proposal Presentation

Literature REVIEW Contd. Paper/ Thesis Comments S. Ravindran, Error Estimates for Reduced Order POD Models of Navier-Stokes Equations, ASME IMECE, 2008, pp. 652-657. A priori error estimate for a proper orthogonal decomposition (POD)-based reduced order model for the Navier-Stokes equations Introduction Ph.D. Proposal Presentation