1. Ph.D. Proposal Presentation
TRANSIENT REDUCED-ORDER CONVECTIVE HEAT TRANSFER MODELING FOR A DATA CENTER
Rajat Ghosh
G.W. Woodruff School of Mechanical Engineering
Georgia Institute of Technology
Atlanta, GA
September 25, 2012

2. Ph.D. Proposal Presentation
Outline
Introduction
Problem Statement
Representative Case Studies
Case study-1
Case Study-2
Case Study-3
Remaining Deliverable
Dissertation Timeline
Closure
Introduction
Ph.D. Proposal Presentation

3. Increasing energy consumption
UB: Upper Bound
LB: Lower Bound
(Based on data reported by J. Koomey in the New York Times, July 31, 2012.)
Need to improve energy efficiency in data centers (DCs).
Introduction
Ph.D. Proposal Presentation

4. Increasing power density
Datacom Equipment Power Trends and Cooling Applications (2005), ASHARE TC 9.9
Frequency of occurance
Need of high-resolution monitoring and feedback control
Both in temporal and spatial dimensions.
Introduction
Ph.D. Proposal Presentation

5. Ph.D. Proposal Presentation
Dynamic data centers
(Liu, J.,Terzis, A., "Sensing data centers for energy efficiency,“Phil. Trans. R. Soc. A (2012))
(CRAC supply temperature data from CEETHERM: Sept. 9-10, 2012, 11 pm -11pm )
Rapidly changing server load leads to dynamic thermal environment
-Dynamic thermal analysis requires fast (near-real-time) modeling algorithm.
-State-of-the-art CFD/HT frameworks are too sluggish.
Need of fast surrogate modeling algorithm
Introduction
Ph.D. Proposal Presentation

6. Ph.D. Proposal Presentation
Data center cooling
1/3 of energy spent in a DC is dedicated to its cooling systems.
Forced convective air cooling:
- Heat generated at chips dissipates via cooling airflow propelled by fans in the computer room air conditioning (CRAC) units.
Various airflow schemes exist:
- Underfloor plenum supply and ceiling return airflow scheme.
Introduction
Ph.D. Proposal Presentation

7. Multiscale thermal system
Turbulent Convection
Turbulent Convection
Turbulent Convection + Conduction
Conduction
Involvement of several decades of length and time scales
Spatial: 5 decades (mm to Dm).
Temporal: 4 decades (10-2 s to 10 s).
Introduction
Ph.D. Proposal Presentation

8. Current Approaches for transient thermal modeling
Model Accuracy
Lumped System Modeling
Computational fluid dynamics/ Heat Transfer (CFD/ HT) Modeling
Reduced-order Modeling
Involves iterative solution of non-linear conservation equations.
Involves posing zero local gradient condition.
Optimal and controllable Trade-off.
Computational Time
Introduction
Ph.D. Proposal Presentation

9. Types of Reduced-order Model (ROM)
Statistical Response Surface Model
Modal Reduction-based
Low- Dimensional Model
Simplified
Physics-based Model
Proper Orthogonal Decomposition (POD/ PCA)
Nonlinear Volterra Theory
Laplacian
Model
Thermal Zone
Model
Harmonic Balance Approximation
Introduction
Ph.D. Proposal Presentation

10. computation for a Transient CFD Simulation
DC Modeling Requirement
m spatial nodes and n time steps.
Restriction on temporal discretization:
The dependent variables for the turbulent convective temperature field: u, v, w, T, ε, k.
Computational step~ O(n(m3 + 4m))
m3: For solving momentum equations together.
4m: For solving pressure correction (continuity)+Temperature + Turbulence
n: Number of time steps
For a rack-level simulation:
m~ 1.4 millions, n~1
t~2 hours in a 5.6 GHz machine
Introduction
Ph.D. Proposal Presentation

11. Computation for a Reduced order modeling
DC Modeling Requirement: m dimensional temperature field with n transient observations.
Initial data is collected via measurements or CFD.
POD/Interpolation-based reduced-order modeling
Computational step~ O(3mn+log(n)+kn+n2+k2))
3mn: Row-wise average + Generation of parameter-dependent component+ Generation of covariance matrix
log(n): Proper orthogonal decomposition of covariance matrix (Power algorithm).
kn: Finding POD coefficients for the input parameter space
n2: Interpolation
k2: Computation of new data (k=principal component number).
No higher power of m.
Introduction
Ph.D. Proposal Presentation

12. Limitation of Existing Modeling Algorithm
Computational fluid dynamic and heat transfer (CFD/HT) modeling
Too sluggish to be fit for a near-real-time modeling algorithm.
Stochastic nature does not warrant expensive CFD simulations.
Reduced-order modeling
A few studies exist with time as the parameter.
No study exists with spatial location as the parameter.
No multi-parameter model exists.
Few studies use experimental data as model input: use of CFD defeats the purpose of using ROM.
Need an alternative to Galerkine projection-based POD coefficient determination.
Problem Statement
Ph.D. Proposal Presentation

13. Ph.D. Proposal Presentation
Scope of dissertation
Development of measurement-based parametric modeling framework
One parameter model
For improving temporal resolution.
For improving spatial resolution.
Multi-parameter model
For improving resolution in an additional dimension like rack heat load.
Development of interconnected multiscale model
- Hybrid reduced-order modeling approach.
Problem Statement
Ph.D. Proposal Presentation

14. single parameter (Time) Reduced-order Algorithm
Proper Orthogonal Decomposition (POD)-based modal reduction.
Time is the modeling parameter.
Reduces the sampling rate
Use interpolation/ extrapolation to determine POD coefficients
Avoid computationally-prohibitive Galerkin projection.
Representative Case Study
Ph.D. Proposal Presentation

15. Case study for single parameter (TIME) POD Model
After remaining shut down for 2 minutes, the CRAC unit is turned on at t=0.
Representative Case Study
Ph.D. Proposal Presentation

16. Optimality of POD Modes
First 10 POD modes capture more than 90% characteristics of the temperature field.
Representative Case Study
Ph.D. Proposal Presentation

17. Principal Component Number
As captured energy percentage increases, the corresponding principal component number increases.
Representative Case Study
Ph.D. Proposal Presentation

18. Ph.D. Proposal Presentation
Error Formulation
Representative Case Study
Ph.D. Proposal Presentation

19. Temperature measurement
Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire.
Response time
20 ms.
Measurement Frequency:
1 Hz.
x-axis: Parallel to rack width.
y-axis: Parallel to tiles.
z-axis: parallel to rack height.
S. Ravindran, Error Estimates for Reduced Order POD Models of Navier-Stokes Equations, ASME IMECE, 2008, pp
Representative Case Study
Ph.D. Proposal Presentation

20. POD/ Interpolation framework
Temperature map at the rack inlet at t=92 s.
A posterior measurement,
t~100 s
An extra step of interpolation,
t~10 s
Deviation~ O(1%)
POD model is efficient in improving parametric resolution of transient temperature data
Accuracy of POD model prediction is identical to experimental data.
Representative Case Study
Ph.D. Proposal Presentation

21. Calibration of analytical error
Calibrated analytical error obviates the necessity of determining a posteriori prediction error .
Representative Case Study
Ph.D. Proposal Presentation

22. POD/ Extrapolation framework
Temperature map at the rack inlet at t=207 s.
A posterior measurement,
t~207 s
An extra step of interpolation,
t~10 s
Deviation~ O(5%)
POD model is efficient in improving parametric resolution of transient temperature data
Accuracy of POD model prediction is identical to experimental data.
Representative Case Study
Ph.D. Proposal Presentation

23. Calibration of analytical Error
Calibrated analytical error obviates the necessity of determining a posteriori prediction error .
Representative Case Study
Ph.D. Proposal Presentation

24. single parameter (SPACE) Reduced-order Algorithm
Proper Orthogonal Decomposition (POD)-based modal reduction.
Coordinates of spatial location are the modeling parameters.
Improves the granularity of experimental data.
Reduction in sensor density.
Representative Case Study
Ph.D. Proposal Presentation

25. Case study for single parameter (Space) POD Model
Sudden shut down of the CRAC unit and power back after 100 s.
(Photo courtesy to IBM)
Representative Case Study
Ph.D. Proposal Presentation

26. Prediction for DOF-1 Points
Improves spatial resolution between (70, 51, -1) and (70, 50, -1).
Representative Case Study
Ph.D. Proposal Presentation

27. Prediction for Dof-2 points
Improves spatial resolution between (56, 31, 2.5) and (56,30,5.5).
Representative Case Study
Ph.D. Proposal Presentation

28. TWO parameter Reduced-order Algorithm
POD-based modal decomposition.
Time and rack heat load as the modeling parameters.
Representative Case Study
Ph.D. Proposal Presentation

29. Case study for multi-parameter (Time, rack heat load) POD Model
Sudden shut down of the CRAC unit and power back after 100 s (t=0 in the plot).
Representative Case Study
Ph.D. Proposal Presentation

30. Comparison for Extrapolation at Q=1500 W
Extrapolation in time and interpolation in heat load.
Representative Case Study
Ph.D. Proposal Presentation

31. Interconnected Multiscale modeling
Experimentally validated CFD/HT modeling for a selected part of the CEETHERM data center laboratory.
Development of the hybrid modeling framework combining finite network modeling (FNM) and POD for simulating a selected part of the CEETHERM data center laboratory.
Comparison and validation.
Deliverable
Ph.D. Proposal Presentation

32. Dissertation TimeLine
May 2010-Dec. 2011
Single-parameter POD framework development with time as the parameter.
Jan June 2012
Single-parameter POD framework development with spatial location(s) as the parameter(s).
June 2012-Oct. 2012
Multi-parameter POD framework development with spatial location and time as the parameters.
Nov April 2013
Interconnected multi-scale modeling.
Planning
Ph.D. Proposal Presentation

33. Ph.D. Proposal Presentation
publications
REFEREED JOURNAL PUBLICATION
Rajat Ghosh, and Yogendra Joshi, “Error Estimate in POD-based Dynamic Reduced-order Thermal Modeling of Data Centers,” International Journal of Heat and Mass Transfer (Revised version submitted).
REFEREED CONFERENCE PUBLICATIONS
Rajat Ghosh, Levente Klein, Yogendra Joshi, and Hendrik Hamann, “Reduced-order Modeling Framework for Improving Spatial Resolution of the Temperature Data Measured in an Air-cool Data Center,” Semi-Therm, San Jose, California, March 17-21,
Rajat Ghosh, Vikneshan Sundaralingam, and Yogendra Joshi, “Effect of Rack Server Population on Temperatures in Data Centers,” Intersociety Thermal Conference (ITherm), San Diego, California, May 30-June 1,
Rajat Ghosh, Vikneshan Sundaralingam, Steven Isaacs, Pramod Kumar and Yogendra Joshi, “Transient Air Temperature Measurements in a Data Center,” Indian Society of Heat and Mass Transfer Conference, Chennai, India, Dec , 2011.
Rajat Ghosh, Pramod Kumar, Vikneshan Sundaralingam, and Yogendra Joshi, “Experimental Characterization of Transient Temperature Evolution in a Data Center Facility,” International Symposium on Transport Phenomena, Delft, the Netherlands, Nov. 8-11, 2011.
Rajat Ghosh and Yogendra Joshi, “Dynamic Reduced-order Thermal Modeling of Data Center Air Temperatures,” InterPACK, Portland, Oregon, July 6-8, 2011.
Planned PUBLICATIONS
Closure
Ph.D. Proposal Presentation

34. Ph.D. Proposal Presentation
Acknowledgement
The support for this work from IBM Corporation, with Dr. Hendrik Hamann as the Technical Monitor, is acknowledged . Acknowledgements are also due to the United States Department of Energy as the source of primary funds. Additional support from the National Science Foundation award CRI enabled the acquisition of some of the test equipment utilized.
The support from the G.W. Woodruff School of Mechanical Engineering as a Graduate Teaching Assistant is acknowledged.
The collaboration, goodwill, and help received from all CEETHERM and METTL members (particularly Vikneshan Sundaralingam, Vaibhav Arghode, Pramod Kumar, Steven Isaacs) are highly appreciated.
Closure
Ph.D. Proposal Presentation

35. Ph.D. Proposal Presentation
Thank YOU!
closure
Ph.D. Proposal Presentation

36. Ph.D. Proposal Presentation
Appendix
Introduction
Ph.D. Proposal Presentation

37. Ph.D. Proposal Presentation
Introduction
Ph.D. Proposal Presentation

38. Ph.D. Proposal Presentation
Introduction
Ph.D. Proposal Presentation

39. Ph.D. Proposal Presentation
Introduction
Impact of proliferated cloud computing-based e-commerce services on data Centers:
Increasing dynamic characteristics.
Increasing energy consumption.
Increasing power densities of racks.
Effect on cooling
30%-40% energy consumed by cooling systems.
Importance of local thermal characteristics.
Need
High resolution (space/time) temperature monitoring.
Near-real-time feedback control for temperature.
Introduction
Ph.D. Proposal Presentation

40. Temperature measurement
Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire.
Response time
20 ms.
Measurement Frequency:
1 Hz.
x-axis: Parallel to rack width.
y-axis: Parallel to tiles.
z-axis: parallel to rack height.
Introduction
Ph.D. Proposal Presentation

41. Dissertation time line
Development of a single-parameter POD-based framework for transient convective heat transfer modeling for an air-cool data center (May 2010-Dec. 2011).
Development of a grid-based thermocouple network for transient air temperature measurements (Nov Nov. 2011).
Development of design protocol for filling out an empty rack (Nov Dec. 2012).
Development of a single-parameter POD-based framework capable of improving spatial resolution of transient temperature data (Mar June 2012).
Development of a two-parameter POD-based framework for transient convective heat transfer modeling for an air-cool data center (May 2012-Dec. 2012).
Development of a scale-linking across various length-scales in a data center (Oct Mar. 2013).
Ph.D. dissertation defense (Mar. 2013).
Introduction
Ph.D. Proposal Presentation

42. Ph.D. Proposal Presentation
May 2010-Dec. 2010
Jan June 2011
July 2011-Dec. 2011
Jan June 2012
June 2012-Dec. 2012
Jan May 2013
Single-parameter POD Framework Development
Introduction
Ph.D. Proposal Presentation

43. Literature REVIEW Paper/ Thesis Comments
S. V. Patankar, Airflow and Cooling in a Data Center, Journal of Heat Transfer 132 (2010)
CFD/HT simulation for a steady data center.
A.H. Beitelmal, C.D. Patel, Thermo-Fluids Provisioning of a High Performance High Density Data Center, Distributed and Parallel Databases, 21, 227–238, 2007.
CFD/HT simulation for a transient data center.
Shawn Shields, Dynamic Thermal Response of the Data Center to Cooling Loss during Facility Power Failure, Masters Thesis, Georgia Tech, 2009.
Measurement-based transient characterization of a data center.
J. D. Rambo, Reduced-order Modeling of Multiscale Turbulent Convection: Application to Data Center Thermal Management, Ph.D. Dissertation, Georgia Tech, 2007.
Reduced-order modeling of data center and a posterior error analysis.
Introduction
Ph.D. Proposal Presentation

44. Literature REVIEW Contd.
Paper/ Thesis
Comments
Q. Nie, Experimentally Validated Multiscale Thermal Modeling of Electronic Cabinets , Ph.D. Dissertation, Georgia Tech, 2008.
Interconnected multiscale modeling for a rack.
Graham Nelson, Development of an Experimentally-Validated Compact Model of a Server Rack, Masters Thesis, Georgia Tech, 2009.
Development of grid-based temperature measurement system and compact modeling of a server.
E. Samadiani, Energy Efficient Thermal Management of Data Centers via Open Multi-scale Design, Ph.D. Dissertation, Georgia Tech, 2009.
Reduced-order open design for a multi-scale data center.
V. López and H. F. Hamann, Heat transfer modeling in data centers, International journal of heat and mass transfer 54 (2011)
Simplified Physics-based Laplacian Model for a data center.
Introduction
Ph.D. Proposal Presentation

45. Literature REVIEW Contd.
Paper/ Thesis
Comments
S. Ravindran, Error Estimates for Reduced Order POD Models of Navier-Stokes Equations, ASME IMECE, 2008, pp
A priori error estimate for a proper orthogonal decomposition (POD)-based reduced order model for the Navier-Stokes equations
Introduction
Ph.D. Proposal Presentation