Reduced-order Modeling Framework for Improving Spatial Resolution of Data Center Transient Air Temperatures Rajat Ghosh, Yogendra Joshi Georgia Institute of Technology 801 Ferst Drive Atlanta, GA Levente Klein, Hendrik Hamann IBM TJ Watson Research Center 1101 Kitchawan Road Yorktown Heights, NY SEMI-THERM 29 March 21, 2013

Dynamic Events in Data Centers 2 Fluctuating IT load Courtesy Junwei Li, CERCS, GT Liu et al., Phil. Trans. R. Soc. A Microsoft Live Messenger Power Outage

Dynamic Resource Allocation 3 Armbrust et al., 2009, Report UCB/EECS Loss of cooling resources ( Lower CRAC set points than required) Over-Provisioning Need for real-time datacenter thermal characterization for better capacity planning

Outline Problem Statement. Methodology. Case Study. Conclusion. 4

Optimization Problem 5 Efficient CRAC control system  Return/ supply air temperature control based on air temperature field. Requirement  Rapid dynamic characterization of DC air temperature field.  Highly-resolved air temperature prediction in time and space. Time scale:10 s 10 kW IT rack 800 W/ ft 3 heat load Length scale: 1” / 2.5 cm

Potential Solution 6 Computational Modeling CFD/ HT-based solution. Discretization of the domain into grid points. Iterative solution of discretized conservation equations. Experiment Deployment of sensor network. Data acquisition. Measurement-based Modeling Using sensor data as input to statistical modeling framework. Data compression techniques: Proper orthogonal decomposition (POD). Multivariate interpolation.

Example Problem 7 A 2 ft. x 2 ft. x 6ft. 10 kW Rack Computational simulation 1.4 grid points. 8 hr. (Quad-core processor and 12 GB RAM) for convergence. Experiment 6” resolution. Difficulty in sensor deployment in largely space-constraint facility. Measurement-based Modeling A platform for improving granularity of sensor data. 2 decades of length scale faster than CFD modeling.

Interpolation Vs. POD Data Matrix: m x n Interpolation: –Computation ~ O(m) POD: –Computation ~ log(k) : k<m. POD Coefficient determination: –Column wise interpolation with a k x n base matrix. –Base matrix elements smaller than data Smaller error due to interpolation. Advantages of POD: –Computationally more efficient. –Better accuracy. 8

Modeling Algorithm 9  Independent Variable Time. Row-wise compilation in ensemble.  Parameter Spatial location. Column-wise compilation in ensemble.  POD Modes Optimal basis.  POD Coefficients Spatial dependency of interrogation.  Principal Component Cut-off Criteria.  Useful tool for analyzing time signals of high dimensionality.

Ensemble Compilation Interrogation Points Two-point method – Two transient temperature data (vector) constitutes the ensemble. – Least data acquisition cost. – Two near most sensors are reasonable choice. – 1-D spatial prediction. 10 Class-1: - Two nearest sensors lying in opposite direction. Class-2: - Two nearest sensors lying in same direction.

1 2 3 Experimental Facility Grey blocks: IT rack. Blue blocks: ACUs. Yellow blocks: PDU. Red block: Storage m 3 /s (12400 cfm ) 2075 W 2 ft. X1.8 ft. X 6 ft W 2 ft. X2 ft. X 6 ft W 2 ft. X2.5 ft. X 6 ft.

Temperature Measurement Sensor # Height mm. (ft.) (7.5) (6.6) (5.8) (5) (4.2) (3.3) (2.5) (1.6) (0.8) 10 0(0)  10 K-type thermocouples placed on a pole, located at the server outlets.  Measurement period: 1.5 s.  Measurement uncertainty: 12 x

Experimental Condition 13 Simulated dynamic temperature field: Periodic blocking and unblocking of rack airflow intake. Photograph Courtesy to Dr. Levente Klein, IBM For this case study, the block/ unblocking period is 30 min.

Data 14 h Boundary effect dominant h In-phase with blocking/ unblocking h h h h h Boundary Effect Appears h h Significant phase shift due to boundary effect (cold air mixing)

Data Comparison No particular temperature trend is observed Maximum at 3.3 ft. Minimum at 7.5 ft. 15

POD-based Prediction For Validation purpose, POD-based predictions are computed at points coincident with the sensors. Two point ensemble compilation method used : 16 Ensemble Sensor # Height mm. (ft.) (2, 3) 2286 (7.5) (1,3) 2012 (6.6) (2,4) 1768 (5.8) (3,5) 1524 (5) (4,6) 1280 (4.2) (5,7) 1006 (3.3) (6,8) 762 (2.5) (7,9) 488 (1.6) (8,10) 244 (0.8) (8,9) 0(0) Interrogation Point Ensemble sensor data Class-2 Ensemble sensor data Interrogation Point Class-1

Interrogation Point: 3.3 ft. (1006 mm). 17 POD-based Modeling Ensemble Sensor: (5,7). Data Matrix: 4425 x 2 Eigen Space 1 st POD mode captures dominant characteristics. Prediction Computational prediction time for a new temperature data ~ 1 s (2.66 GHz Core2Duo processor, 4 GB RAM). k=1: only 2 interpolations required.

Comparison 18

19 Error Distribution Time Sample Size=4425. Large error due boundary effect

Space-time Mapping 20 Increase in Temperature due to Blocking Decrease in Temperature due to Unblocking Large Error at h=0 due the Boundary Effect

Conclusion A modeling framework is developed for improving the spatial resolution of experimentally-acquired transient temperature data. 21 The framework is applied on a representative case study with dynamic temperature evolution. The framework predicts the temperature evolution with reasonable accuracy.

22