July 13, GrenchMark: A workload generator for Grids Demo at TU Delft A. Iosup, D.H.J. Epema PDS Group, ST/EWI, TU Delft

July 13, Evaluating Grid schedulers performance Performance Qualitative metrics supported application types, advanced fault tolerance, advanced... Quantitative metrics resource consumption, system performance, scheduler functionality Other metrics cost, single number Needs Applications, workloads, more workloads… Benefits Configuration testing and tuning Functionality testing System design and procurement Performance testing Runtime estimation

July 13, GrenchMark A systematic approach to testing Grid schedulers A set of metrics for comparing schedulers A set of representative applications Both real and synthetic Sequential tasks, MPI tasks, Java, IBIS/Satin tasks, batch tasks, filters/streaming tasks, chains of tasks, DAGs, general graphs,...(?) Easy tools to create synthetic workloads [TODAY] Testing, testing, testing… [EVERY DAY]

July 13, The GrenchMark process

July 13, Outline Introduction Grenchmark Using Grenchmark Customizing Grenchmark Extending Grenchmark Conclusions [done] [here]

July 13, wl-desc.in Describes the workload to be generated; input file for wl-gen.py Fixed, non-fixed, semi-fixed execution sites Applications inter-arrival time: Poisson, uniform, normal, exp, hyperexp2, Weibull, log normal, gamma, … Over 35 supported applications, in 3 categories: sequential, MPI, Ibis

July 13, Grenchmark process $./wl-gen.py wl-desc.in $./wl-submit.py out/wl-to-submit.wl TODO Sample run: 95 jobs / 411 components 95 files / 32 directories 300KB data Sample run: multi-threaded – submission delay +/- 0.01s 95 JDFs

July 13, Preliminary notions Job, workload, workload unit Job = Set of components (co-allocation support) [Job = one program execution / the basic scheduling unit / … ] Workload = Set of Jobs Workload Unit = Set of jobs with the same property, generated in the same step (definition useful only for workload generator) Other JDF = Job description file Inter arrival time = time between two consecutive jobs arrive in the system

July 13, Describing the workload: Jobs wl-desc.in ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) - Generate Times x Jobs or Times x Sets of Jobs One Job = k Components (here: 1 -- single)

July 13, Demo 1: Generating one job Single site, single instance

July 13, Describing the workload: Jobs wl-desc.in ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) - Generate Times x Jobs or Times x Sets of Jobs One Job = k Components (here: 1 -- single)

July 13, Describing the workload: Units wl-desc.in ID Times UnitType SiteType TotalComps PerSiteDesc ArrivalTimeDistr OtherInfo ? 1 wl:poisson Poisson(500) File=wl.xin wl.xin RandomWorkload=false NComponents=4;8 ComponentSize=8;16 Submitter=drunner Application=pi-g2-gm Generate Times x Jobs or Times x Sets of Jobs wl: prefix – workload unit generator [Hashim’s jobs]

July 13, Demo 4: Generating one workload unit Hashim’s sample workload: Poisson-g2-gm (Globus MPICH-G2) Components: 4 and 8 Component Sizes: 8 and 16 IAT Poisson(5s) Run with drunner Total: 4 jobs, 24 components

July 13, Demo 5: Generating a full workload Hashim’s full workload, 8 times: Poisson-g2-gm (Globus MPICH-G2) and findpi-gm Components: 4 and 8 Component Sizes: 4/8 and 16 IAT Poisson(5s) Run with drunner (poisson) / krunner (pi) Submit time 1 day Generate: Submit: Total: 3200 jobs, components / 3k files, 4k dirs Timing: 30s generate / 86,400s submit (1 day, eh?) $ wl-gen.py --duration= wl-desc.in $ wl-submit.py --onefile out/wl-desc.in

July 13, Outline Introduction Grenchmark Using Grenchmark Customizing Grenchmark Extending Grenchmark Conclusions [done] [here] [done]

July 13, Customizing the workload generator Command-line parameters wl-gen.py –help … displays a comprehensive help screen wl-gen.py –o outdir … creates the workload in directory outdir wl-gen.py –d … sets the total submission duration to 50s wl-gen.py –j my-jdf … uses my-jdf to write JDFs wl-gen.py –o out-1 wl-desc.in my favourite command

July 13, Customizing the submission tool Command-line parameters wl-submit.py –help … displays a comprehensive help screen wl-submit.py –t 10 … uses 10 threads to submit wl-submit.py --nosubmit … just prints the submission commands wl-submit.py --onefile … output job submit (runner) stderr and stdout to one file each (concatenated) wl-submit.py out/wl-to-submit.wl my favourite command

July 13, Outline Introduction Grenchmark Using Grenchmark Customizing Grenchmark Extending Grenchmark Conclusions [done] [here] [done]

July 13, Extending the workload generator (1) Write your own Job Generators 1.Create your-job-generator.py in apps/ 2.Place a Generator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example: Generator=""" Name=your-job-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“ 3.Create a function generateWorkloadUnit, which takes in UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom 4.Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each 5.Use the generator in wl-desc.in by putting your-job-type in the UnitType field

July 13, Extending the workload generator (2) Write your own Unit Generators 1.Create your-job-generator.py in apps/ 2.Place a WLGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example: WLGenerator=""" Name=your-workload-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“ 3.Create a function generateWorkload, which takes in UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom 4.Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each 5.Use the generator in wl-desc.in by putting your-job-type in the UnitType field

July 13, Extending the workload generator (2) Unit Generators in C, Ruby, Perl,... ? 1.Same as previous: generate your-job-generator.py in apps/ 2.Replace step 4 with: 1.Interface your Python code, so that it runs your C, C++, Ruby, Perl, … application (os.exec, os.popen, …) 2.Save your C, C++, Ruby, Perl, … application information to a text file, or use pipes to transfer info 3.Read the saved information.

July 13, Extending the workload generator (3) Write your own JDF Writers 1.Create your-jdf-writer.py in jdfprint/ 2.Place a JDFGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example: JDFGenerator=""" Name=your-jdf-writer-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“ 3.Create a function generateJDF, which takes in OutFileName, ListOfComponents 4.Make generateJDF write the components to the specified file, with appropriate data each 5.Use the generator by calling wl-gen.py with the –j parameter

July 13, Conclusions and future work Grenchmark easily generates diverse workloads of Grid applications KOALA developers successfully used GrenchMark to test KOALA Future work do you have specific Grid applications types that you would like to test? Try Grenchmark!

July 13, Thank you! Questions? Remarks? Observations? All welcome! Grenchmark [10x Paulo] Alexandru IOSUP TU Delft [google: “iosup”]