1. SC05 November, 2005 clyne@ncar.ucar.edu Desktop Techniques for the Exploration of Terascale Sized Turbulence Data Sets John Clyne Scientific Computing Division National Center for Atmospheric Research Boulder, CO USA

2. SC05 November, 2005 clyne@ncar.ucar.edu [Numerical] models that can currently be run on typical supercomputing platforms produce data in amounts that make storage expensive, movement cumbersome, visualization difficult, and detailed analysis impossible. The result is a significantly reduced scientific return from the nation's largest computational efforts. We can now compute more data than we know how to analyze!!!

3. SC05 November, 2005 clyne@ncar.ucar.edu A sampling of various technology performance curves Not all technologies advance at same rate!!!

4. SC05 November, 2005 clyne@ncar.ucar.edu

5. SC05 November, 2005 clyne@ncar.ucar.edu

6. SC05 November, 2005 clyne@ncar.ucar.edu Growth of individual NCAR simulation data sets Representative data sets from climate, turbulence, and weather disciplines

7. SC05 November, 2005 clyne@ncar.ucar.edu Climate simulation grid resolutions

8. SC05 November, 2005 clyne@ncar.ucar.edu Example: Compressible plume dynamics 504x504x2048 5 variables (u,v,w,rho,temp) ~500 time steps saved 9TBs storage (4GBs/variable/timestep) Six months compute time required on 112 IBM SP RS/6000 processors Three months for post-processing Data may be analyzed for several years M. Rast, 2004. Images courtesy of Joseph Mendoza, NCAR/SCD

9. SC05 November, 2005 clyne@ncar.ucar.edu Workflow in computational science Simulation Analysis & Visualization Storage Post Processing Storage Batch Batch & Interactive Interactive Bandwidth requirements?

10. SC05 November, 2005 clyne@ncar.ucar.edu What is meant by interactive computing? Definition: A system is interactive if the time between a user event and the response to that event is short enough maintain my full attention If the response time is… 1-5 seconds : I’m engaged 5-60 seconds : I’m reading email 1-3 minutes : I’ve forgotten what I was trying to do > 3 minutes : I’ve given up!

11. SC05 November, 2005 clyne@ncar.ucar.edu IO wait times for high resolution simulations Assumptions –Single precision –100 MB/sec bandwidth –No contention ResolutionMBs per variable Scalar variable wait time Vector variable wait time 128 3 80.10.3 256 3 670.72.1 512 3 5375.015.0 1024 3 429543.0130.0 Interactive! Reading mail!!

12. SC05 November, 2005 clyne@ncar.ucar.edu Visualization and Analysis Platform for oceanic, atmospheric, and solar Research (VAPoR) Key components 1.Domain specific application focus: numerically simulated turbulence 2.Quantitative capabilities to support scientific data analysis 3.Integrate visualization into analysis process, interactively steering the analysis while enhancing data understanding 4.Employ multiresolution data representation as a data reduction technique This work is funded in part through a U.S. National Science Foundation, Information Technology Research program grant Combination of visualization with multiresolution data representation that provide sufficient data reduction to enable interactive work

13. SC05 November, 2005 clyne@ncar.ucar.edu Enabling speed/quality tradeoffs with multiresolution data representation 1 Multiple copies of data at varying power of two resolutions Storage costs: 1/2 1/4 1/8 Example: Texture MIP Mapping2D Example: Texture MIP Mapping

14. SC05 November, 2005 clyne@ncar.ucar.edu Wavelet Transforms for 3D Multiresolution data representation Permit hierarchical data representation Invertible and lossless (subject to floating point round off errors) Numerically efficient – forward and inverse transform No additional storage cost!!!

15. SC05 November, 2005 clyne@ncar.ucar.edu Compressible Convection 128 3 512 3 M. Rast, 2002

16. SC05 November, 2005 clyne@ncar.ucar.edu 504x504x2048 Full 252x252x1024 1/8 126x126x512 1/64 63x63x256 1/512 Compressible plume data set shown at native and progressively coarser resolutions Compressible plume Resolution: Problem size:

17. SC05 November, 2005 clyne@ncar.ucar.edu Rendering timings 512 3 Compressible Convection504 2 x2048 Compressible Plume Reduced resolution affords responsive interaction while preserving all but finest features SGI Octane2, 1x600MHz R14k SGI Origin, 10x600MHz R14k Interactive

18. SC05 November, 2005 clyne@ncar.ucar.edu Derived quantities p:pressure  :density T:temperature   :ionization potential   :Avogadro’s number m e :electron mass k:Boltzmann’s constant h:Planck’s constant Derived quantities produced from the simulation’s field variables as a post- process

19. SC05 November, 2005 clyne@ncar.ucar.edu Calculation timings for derived quantities Note: 1/2 th resolution is 1/8 th problem size, etc Deriving new quantities on interactive time scales only possible with data reduction SGI Origin, 10x600MHz R14k

20. Integrated visualization and analysis on interactively selected subdomains: Vertical vorticity of the flow. Mach number of the vertical velocity. Full domain seen from above.Subdomain from side. Full domain seen from above.Subdomain from side. Efficient analysis requires rapid calculation and visualization of unanticipated derived quantities. This can be facilitated by a combination of subdomain selection and resolution reduction.

21. A test of multiresolution analysis: Force balance in supersonic downflows Sites of supersonic downflow are also those of very high vertical vorticity. The cores of the vortex tubes are evacuated, with centripetal acceleration balancing that due to the inward directed pressure gradient. Buoyancy forces are maximum on the tube periphery due to mass flux convergence. The same interpretation results from analysis at half resolution. Full Half Resolution Subdomain selection and reduced resolution together yield data reduction by a factor of 128!!!

22. SC05 November, 2005 clyne@ncar.ucar.edu Future??? Original20:1 Lossy Compression

23. SC05 November, 2005 clyne@ncar.ucar.edu Live VAPOR demonstrations, SGI Theatre (booth # 602): –Wednesday, 11:30am –Thursday, 3:30pm VAPOR URL: –http://www.scd.ucar.edu/hss/dasg/software/vapor Questions???

24. SC05 November, 2005 clyne@ncar.ucar.edu Inadequate IO bandwidth is but one impediment to interactive analysis and visualization. Others impediments include: –Insufficient capacity of high-speed storage –Reliance on un-optimized, serial applications –Mismatch between simulation and analysis computing resources

25. SC05 November, 2005 clyne@ncar.ucar.edu NCAR Science Space Weather Turbulence Atmospheric Chemistry Climate Weather The Sun More than just the atmosphere… from the earth’s oceans to the solar interior