ASCR Scientific Data Management Analysis & Visualization PI Meeting Exploration of Exascale In Situ Visualization and Analysis Approaches LANL: James Ahrens, Jon Woodring, Joanne Wendelberger, Francesca Samsel. We explore two in situ approaches at the extreme ends of a spectrum between flexibility and accuracy. We will strive to understand the advantages and disadvantages of both approaches and evaluate their effectiveness. Using the results of this evaluation, we will merge the best of both approaches to produce an optimize exascale visualization and analysis approach. Statistics and Sampling of Simulation Data with Bitmaps Challenges  Locating the data that a scientist needs is daunting due to the scale of the data and lack of information Solution: Sample Bitmaps  Bitmap indices provide summary information for a large-scale data set  They also provide distributional data that can be used for sampling  Statistics can be extracted from this summary to be able to drill down and extract information of interest  Bitmaps accelerate statistics and sampling for faster turn-around in exploration with lower sample error Papers  Y. Su, G. Agrawal, J. Woodring, K. Myers, J. Wendelberger and J. Ahrens, "Effective and Efficient Data Sampling Using Bitmap Indices", Cluster Computing, March  Y. Su, G. Agrawal, J. Woodring, A. Biswas and H.-W. Shen, "Supporting Correlation Analysis on Scientific Datasets in Parallel and Distributed Settings", in Proceedings of the International ACM Symposium on High- Performance Parallel and Distribued Computing (HPDC'14), June 2014, Vancouver, Canada.  Y. Su, G. Agrawal, J. Woodring, K. Myers, J. Wendelberger and J. Ahrens. “Taming Massive Distributed Datasets: Data Sampling Using Bitmap Indices.” In Proceedings of the International ACM Symposium on High- Performance Parallel and Distributed Computing (HPDC’13), New York, NY, USA, June  Y. Su, G. Agrawal, and J. Woodring, “Indexing and Parallel Query Processing Support for Visualizing Climate Datasets”, Proceedings of the 41st International Conference on Parallel Processing, Pittsburgh, PA, Sept Bitmaps are used for sampling and statistics for large-scale data analysis Contact: James Ahrens Adaptive refinement based on analysis metric highlighting areas of interest Reduced Simulation Data Approach Significantly reducing simulation data by storing sampled and compressed data representations Adaptive Sampling of Simulation Data Challenges  Simulations and experiments generate more data that can be feasibly stored by the scientist Solution: Adaptive Sample Data based on Analysis Metrics  Treat the exascale data deluge as a sampling problem  Use a variety of metrics to automatically select and triage the important data  Analysis Driven Refinement is a framework that prioritizes and samples using these metrics Papers  B. Nouanesengsy, J. Woodring, K. Myers, J. Patchett, and J. Ahrens, “ADR Visualization: A Generalized Framework for Ranking Large-Scale Scientific Data using Analysis-Driven Refinement”, LDAV 2014, November 2014, Paris, France.  K. Myers, E. Lawrence, M. Fugate, J. Woodring, J. Wendelberger, and J. Ahrens, “An In Situ Approach for Approximating Complex Computer Simulations and Identifying Important Time Steps”, in submission, arXiv:  A. Biswas, S. Dutta, H.-W. Shen, J. Woodring. “An Information-Aware Framework for Exploring Multivariate Data Sets.” IEEE Visualization 2013, Atlanta, GA, November, Image Database Approach Significantly reducing simulation data by storing rendered visualization and analysis images into an image database Sampling in “Visualization and Analysis” Space Challenges  Simulations and experiments generate massive datasets that are difficult to store and analysis in a post processing manner Solution: Generate In Situ Image Database  Enables many different interaction modes including: 1) animation and selection, 2) camera and 3) time  Creates an responsive interactive visualization solution, rivaling modern post-processing approaches, based on producing constant time retrieval and assembly  Encourages the use of both computationally intensive analysis and temporal exploration typically avoided in post-processing approaches Supports Metadata Searching  By leveraging an image database, our approach allows the analyst to execute meta-data queries or browse analysis results to produce a prioritized sequence of matching results Creation of New Visualizations and Content Querying  Supports composing of individually imaged operators  Provides access to the underlying data to enable advanced rendering during post-processing (e.g. new lookup tables, lighting,...)  Makes it possible to perform queries that search on the content of the image in the database. Using image-based visual queries, the analyst can ask simple scientific questions and get the expected results Papers  J. Ahrens, S. Jourdain, P. O'Leary, J. Patchett, D. H. Rogers, M. Petersen, “An Image- based Approach to Extreme Scale In Situ Visualization and Analysis”, Supercomputing 2014, New Orleans. Interactive visualization and compositing using images from the image database Using lighting and color mapping, render passes and compositing enable more capable visualization pipelines such as changing color scale mapping for objects Queries based on the image content can be used to search for qualitative results like “best view” LA-UR