1. VisIt: a visualization tool for large turbulence simulations  Outline Success stories with turbulent simulations Overview of VisIt project 1 Hank Childs Lawrence Berkeley National Laboratory, USA April 30 th, 2010 NORDITA and Linné FLOW Centre Workshop on Turbulent Boundary Layers

2. 2 27B element Rayleigh-Taylor Instability (MIRANDA, BG/L) VisIt: a visualization tool for large turbulence simulations Large data requires special techniques. This image was made by using a distributed memory parallel program that partitioned the data over the processors, performing I/O, processing, and rendering in parallel.

3. 3 VisIt calculated connected components on a 4K^3 turbulence data in parallel using TACC's Longhorn machine. 2 million components were initially identified and then the map expression was used to select only the components that had total volume greater than 15. Data courtesy of P.K. Yeung & and Diego Donzis Visualizing and Analyzing Large-Scale Turbulent Flow  Detect, track, classify, and visualize features in large-scale turbulent flow.

4. 4 Flow analysis for 217 pin simulation / 1 billion grid points 217 pin reactor cooling simulation. Run on ¼ of Argonne BG/P.

5. 5 Flow analysis for 217 pin simulation / 1 billion grid points

6. 6 Tracing particles through the channels

7. 7 Place 1000 particles in one channel Observe which channels the particles come out -Very difficult to parallelize particle advection: -Parallelizing over data: load balancing -Parallelizing over particles: redundant I/O -VisIt has a best-in-class algorithm that balances between these extremes.

8. 88 4 VisIt is a richly featured, turnkey application for very large data VisIt is an open source, end user visualization and analysis tool for simulated and experimental data –Used by: physicists, engineers, code developers, vis experts –>100K downloads on web R&D 100 award in 2005 Used “heavily to exclusively” on 8 of world’s top 12 supercomputers 217 pin reactor cooling simulation. Run on ¼ of Argonne BG/P. 1 billion grid points

9. 9 Terribly Named! Intended for more than just visualization… Data Exploration Visual Debugging Quantitative Analysis Presentations = ? Comparative Analysis

10. 10 VisIt has a rich feature set that can impact many science areas.  Meshes: rectilinear, curvilinear, unstructured, point, AMR  Data: scalar, vector, tensor, material, species  Dimension: 1D, 2D, 3D, time varying  Rendering (~15): pseudocolor, volume rendering, hedgehogs, glyphs, mesh lines, etc…  Data manipulation (~40): slicing, contouring, clipping, thresholding, restrict to box, reflect, project, revolve, …  File formats (~85)  Derived quantities: >100 interoperable building blocks +,-,*,/, gradient, mesh quality, if-then-else, and, or, not  Many general features: position lights, make movie, etc  Queries (~50): ways to pull out quantitative information, debugging, comparative analysis

11. 11 VisIt employs a parallelized client-server architecture.  Client-server observations: Good for remote visualization Leverages available resources Scales well No need to move data  Additional design considerations: Plugins Multiple UIs: GUI (Qt), CLI (Python), more… remote machine Parallel vis resources User data localhost – Linux, Windows, Mac Graphics Hardware

12. 12 The VisIt team focuses on making a robust, usable product for end users.  Manuals 300 page user manual 200 page command line interface manual “Getting your data into VisIt” manual  Wiki for users (and developers)  Revision control, nightly regression testing, etc  Executables for Windows, Mac, Linux, & many Unix  Day long class, complete with exercises Slides from the VisIt class

13. 13 VisIt is a vibrant project with many participants.  Over 50 person-years of effort  Over one million lines of code  Partnership between: Department of Energy’s Office of Nuclear Energy, Office of Science, and National Nuclear Security Agency, and among others Many emerging relationships, including NSF XD centers 2004-6 User community grows, including AWE & ASC Alliance schools Fall ‘06 VACET is funded Spring ‘08 AWE enters repo 2003 LLNL user community transitioned to VisIt 2005 2005 R&D100 2007 SciDAC Outreach Center enables Public SW repo 2007 Saudi Aramco funds LLNL to support VisIt Spring ‘07 GNEP funds LLNL to support GNEP codes at Argonne Summer‘07 Developers from LLNL, LBL, & ORNL Start dev in repo ‘07-’08 UC Davis & UUtah research done in VisIt repo 2000 Project started ‘07-’08 Partnership with CEA is developed 2008 Institutional support leverages effort from many labs Spring ‘09 More developers Entering repo all the time

14. 14 ● Weak scaling study: ~62.5M cells/core 14 #coresProblem Size ModelMachine 8K0.5TIBM P5Purple 16K1TSunRanger 16K1TX86_64Juno 32K2TCray XT5JaguarPF 64K4TBG/PDawn 16K, 32K1T, 2TCray XT4Franklin VisIt was recently demonstrated to show good performance at unprecedented scales.

15. 15 Three Ways To Get Data Into VisIt  (1) Write to a known output format  (2) Write a plugin file format reader  (3) Integrate VisIt “in situ” “lib-VisIt” is linked into simulation code −(Note: Memory footprint issues with implementation!) Use model: −simulation code advances −at some time interval (e.g. end of cycle), hands control to lib-VisIt. −lib-VisIt performs vis & analysis tasks, then hands control back to simulation code −repeat 15

16. 16 VisIt: What’s the Big Deal?  What’s the big deal? Everything works at scale Robust, usable tool Vis to code development to scientific insight  Learn more: Google: “visit visualization” http://www.llnl.gov/visit / http://www.visitusers.org http://www.llnl.gov/visithttp://www.visitusers.org visitusers@ornl.gov Live & online tutorials Publicly available SVN Hank Childs, −hchilds@lbl.gov 16