GA 1 CASC Discovery of Access Patterns to Scientific Simulation Data Ghaleb Abdulla LLNL Center for Applied Scientific Computing

GA 2 CASC Team Ghaleb Abdulla (0.4) Tina Eliassi-Rad (0.4) Terence Critchlow (0.15)

GA 3 CASC Task Objective Identify data storage formats that minimize access times using historical access patterns to the same or similar data sets Use spatial and temporal locality that result from data accesses to format the data on the disk

GA 4 CASC Challenges Data can be accessed using different: —tools, —by different users.

GA 5 CASC Enabling Access Pattern Discovery Application area (astrophysics) Visualization tool (VisIt) Analyze history of access patterns on two levels: —System Level –Disk references –Network overhead –Memory usage — Application level –Higher level commands –User level info

GA 6 CASC Enabling Access Pattern Discovery VisIt Astrophysics User 1 User n Application Logging Disk Logging Log files Unsupervised Learner (e.g., k-NN, k- means, etc) Supervised Learner (e.g., neural net, DT, etc) Hints [Pattern, Hints] training data Patterns Djehuty

GA 7 CASC Log file collection Collect logs at the application and disk level Managing log collection process —Start and stop collection sensors or agents based on demand —Keep log data in one central place —Detect any failure in the monitoring agents and restart them —Preferably work in a distributed environment JAMM from LBL meets our requirements

GA 8 CASC JAMM Architecture

GA 9 CASC What to Collect Application and user level: —Open —Zoom —Slice —etc. System level —Network overhead —Disk block size —Buffer size —Disk location, etc. We need to add our own sensors to collect data

GA 10 CASC Data format The DTD for our XML files is as follows: <!ATTLIST metadata name ID #REQUIRED time NMTOKENS #IMPLIED>

GA 11 CASC Log File, Example 100K K write random 210M 128K

GA 12 CASC Data Analysis Researched publicly available clustering tools Narrowed our choice to two —CLUTO (University of Minnesota) — R (GNU) Testing data processing algorithms on randomly generated log files Hoping to get real log files in the near future: — Logging applications —We are currently looking at the “Flash” Log files

GA 13 CASC Questions

GA 14 CASC This work was performed under the auspices of the U.S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under contract No. W Eng-48. UCRL-MI-xxxxxx