1. Indiana University Faculty Geoffrey Fox, David Crandall, Judy Qiu, Gregor von Laszewski Data Science at Digital Science Center 1

2. Work on Applications Algorithms Systems Software Biology/Bioinformatics Computational Finance Network Science and Epidemiology Analysis of Biomolecular Simulations Analysis of Remote Sensing Data Computer Vision Pathology Images Real time robot data Parallel Algorithms and Software –Deep Learning –Clustering –Dimension Reduction –Image Analysis –Graph 2

3. Digital Science Center Research Areas Digital Science Center Facilities RaPyDLI Deep Learning Environment SPIDAL Scalable Data Analytics Library MIDAS Big Data Software Big Data and HPC Convergence Diamonds Application Classification and Benchmarks CloudIOT Internet of Things Environment Cloudmesh Cloud and Bare metal Automation XSEDE TAS Monitoring citations and system metrics Data Science Education with MOOC’s 3

4. DSC Computing Systems 128 node Haswell based system (Juliet) –128 GB memory per node –Substantial conventional disk per node (8TB) plus PCI based SSD –Infiniband with SR-IOV –24 and 36 core nodes (3456 total cores) Working with SDSC on NSF XSEDE Comet System (Haswell 47,776 cores) Older machines –India (128 nodes, 1024 cores), Bravo (16 nodes, 128 cores), Delta(16 nodes, 192 cores), Echo(16 nodes, 192 cores), Tempest (32 nodes, 768 cores) with large memory, large disk and GPU Optimized for Cloud research and Large scale Data analytics exploring storage models, algorithms Build technology to support high performance virtual clusters 4

5. Cloudmesh Software Defined System Toolkit Cloudmesh Open source http://cloudmesh.github.io/ supportinghttp://cloudmesh.github.io/ –The ability to federate a number of resources from academia and industry. This includes existing FutureSystems infrastructure, Amazon Web Services, Azure, HP Cloud, Karlsruhe using several IaaS frameworks –IPython-based workflow as an interoperable onramp Supports reproducible computing environments Uses internally Libcloud and Cobbler Celery Task/Query manager (AMQP - RabbitMQ) MongoDB Gregor von Laszewski Fugang Wang 5

6. IOTCloud Device  Pub-Sub  Storm  Datastore  Data Analysis Apache Storm provides scalable distributed system for processing data streams coming from devices in real time. For example Storm layer can decide to store the data in cloud storage for further analysis or to send control data back to the devices Evaluating Pub-Sub Systems ActiveMQ, RabbitMQ, Kafka, Kestrel Turtlebot and Kinect 6

7. Crandall 2012 Ground Truth Glacier Beds Snow Radar Lee 2015 7

8. 10 year US Stock daily price time series mapped to 3D (work in progress) 3400 stocks Sector Groupings up One year 2004 Velocities or Positions 8

9. July 21 2007 Positions End 2008 Positions 9

10. End of 2014 Positions 10

11. Jan 27 2012 velocities Jan 1 2015 velocities 11

12. Protein Universe Browser for COG Sequences with a few illustrative biologically identified clusters 12

13. 3D Phylogenetic Tree from WDA SMACOF 13

14. Big Data and (Exascale) Simulation Convergence I Our approach to Convergence is built around two ideas that avoid addressing the hardware directly as with modern DevOps technology it isn’t hard to retarget applications between different hardware systems. Rather we approach Convergence through applications and software. We break applications into data plus model and introduce 64 facets of Convergence Diamonds that describe both Big Simulation and Big Data applications and so allow one to more easily identify good approaches to implement Big Data and Exascale applications in a uniform fashion. The software approach builds on the HPC-ABDS High Performance Computing enhanced Apache Big Data Software Stack concept (http://dsc.soic.indiana.edu/publications/HPC-ABDSDescribed_final.pdf, http://hpc-abds.org/kaleidoscope/ )http://dsc.soic.indiana.edu/publications/HPC-ABDSDescribed_final.pdf http://hpc-abds.org/kaleidoscope/ This arranges key HPC and ABDS software together in 21 layers showing where HPC and ABDS overlap. It for example, introduces a communication layer to allow ABDS runtime like Hadoop Storm Spark and Flink to use the richest high performance capabilities shared with MPI Generally it proposes how to use HPC and ABDS software together. –Layered Architecture offers some protection to rapid ABDS technology change (for ABDS independent of HPC) 14

15. Big Data - Big Simulation (Exascale) Convergence Lets distinguish Data and Model (e.g. machine learning analytics) in Big Data problems Then in Big Data, typically Data is large but Model varies –E.g. LDA with many topics or deep learning has large model –Clustering or Dimension reduction can be quite small Simulations can also be considered as Data and Model –Model is solving particle dynamics or partial differential equations –Data could be small when just boundary conditions or –Data large with data assimilation (weather forecasting) or when data visualizations produced by simulation In each case, Data often static between iterations (unless streaming), model varies between iterations 15

16. 51 Detailed Use Cases: Contributed July-September 2013 Covers goals, data features such as 3 V’s, software, hardware http://bigdatawg.nist.gov/usecases.php https://bigdatacoursespring2014.appspot.com/course (Section 5)https://bigdatacoursespring2014.appspot.com/course Government Operation(4): National Archives and Records Administration, Census Bureau Commercial(8): Finance in Cloud, Cloud Backup, Mendeley (Citations), Netflix, Web Search, Digital Materials, Cargo shipping (as in UPS) Defense(3): Sensors, Image surveillance, Situation Assessment Healthcare and Life Sciences(10): Medical records, Graph and Probabilistic analysis, Pathology, Bioimaging, Genomics, Epidemiology, People Activity models, Biodiversity Deep Learning and Social Media(6): Driving Car, Geolocate images/cameras, Twitter, Crowd Sourcing, Network Science, NIST benchmark datasets The Ecosystem for Research(4): Metadata, Collaboration, Language Translation, Light source experiments Astronomy and Physics(5): Sky Surveys including comparison to simulation, Large Hadron Collider at CERN, Belle Accelerator II in Japan Earth, Environmental and Polar Science(10): Radar Scattering in Atmosphere, Earthquake, Ocean, Earth Observation, Ice sheet Radar scattering, Earth radar mapping, Climate simulation datasets, Atmospheric turbulence identification, Subsurface Biogeochemistry (microbes to watersheds), AmeriFlux and FLUXNET gas sensors Energy(1): Smart grid 16 26 Features for each use case Biased to science

17. Problem Architecture View of Ogres (Meta or MacroPatterns) i.Pleasingly Parallel – as in BLAST, Protein docking, some (bio-)imagery including Local Analytics or Machine Learning – ML or filtering pleasingly parallel, as in bio- imagery, radar images (pleasingly parallel but sophisticated local analytics) ii.Classic MapReduce: Search, Index and Query and Classification algorithms like collaborative filtering (G1 for MRStat in Features, G7) iii.Map-Collective: Iterative maps + communication dominated by “collective” operations as in reduction, broadcast, gather, scatter. Common datamining pattern iv.Map-Point to Point: Iterative maps + communication dominated by many small point to point messages as in graph algorithms v.Map-Streaming: Describes streaming, steering and assimilation problems vi.Shared Memory: Some problems are asynchronous and are easier to parallelize on shared rather than distributed memory – see some graph algorithms vii.SPMD: Single Program Multiple Data, common parallel programming feature viii.BSP or Bulk Synchronous Processing: well-defined compute-communication phases ix.Fusion: Knowledge discovery often involves fusion of multiple methods. x.Dataflow: Important application features often occurring in composite Ogres xi.Use Agents: as in epidemiology (swarm approaches) This is Model xii.Workflow: All applications often involve orchestration (workflow) of multiple components 17

18. 6 Forms of MapReduce cover “all” circumstances Describes - Problem (Model reflecting data) - Machine - Software Architecture 18

19. 19 Green implies HPC Integration Big Data and (Exascale) Simulation Convergence II

20. 20

21. Things to do for Big Data and (Exascale) Simulation Convergence III Converge Applications: Separate data and model to classify Applications and Benchmarks across Big Data and Big Simulations to give Convergence Diamonds with 64 facets –Indicated how to extend Big Data Ogres (50) to Big Simulations by looking separately at model and data in Ogres –Diamonds have four views or collections of facets: Problem Architecture; Execution; Data Source and Style; Processing view covering Big Data and Big Simulation Processing –Facets cover data, model or their combination – the problem or application 16 System Facets; 16 Data Facets; 32 Model Facets –Note Simulation Processing View has similarities to old parallel computing benchmarks 21

22. Things to do for Big Data and (Exascale) Simulation Convergence IV Convergence Benchmarks: we will use benchmarks that cover the facets of the convergence diamonds i.e. cover big data and simulations; –As we separate data and model, compute intensive simulation benchmarks (e.g. solve partial differential equation) will be linked with data analytics (the model in big data) –IU focus SPIDAL (Scalable Parallel Interoperable Data Analytics Library) with high performance clustering, dimension reduction, graphs, image processing as well as MLlib will be linked to core PDE solvers to explore the communication layer of parallel middleware –Maybe integrating data and simulation is an interesting idea in benchmark sets Convergence Programming Model –Note parameter servers used in machine learning will be mimicked by collective operators invoked on distributed parameter (model) storage –E.g. Harp as Hadoop HPC Plug-in –There should be interest in using Big Data software systems to support exascale simulations –Streaming solutions from IoT to analysis of astronomy and LHC data will drive high performance versions of Apache streaming systems 22

23. Things to do for Big Data and (Exascale) Simulation Convergence V Converge Language: Make Java run as fast as C++ (Java Grande) for computing and communication – see following slide –Surprising that so much Big Data work in industry but basic high performance Java methodology and tools missing –Needs some work as no agreed OpenMP for Java parallel threads –OpenMPI supports Java but needs enhancements to get best performance on needed collectives (For C++ and Java) –Convergence Language Grande should support Python, Java (Scala), C/C++ (Fortran) 23

24. Java MPI performs better than Threads I 128 24 core Haswell nodes Default MPI much worse than threads Optimized MPI using shared memory node-based messaging is much better than threads 24

25. Java MPI performs better than Threads II 128 24 core Haswell nodes 25 200K Dataset Speedup

26. Oct 25 2013 velocities 26