June 29, Grenchmark: A workload generator for Grid schedulers First Demo at TU Delft A. Iosup, D.H.J. Epema PDS Group, ST/EWI, TU Delft

June 29, Evaluating Grid schedulers performance Grid schedulers performance Qualitative metrics supported application types, advanced fault tolerance, advanced... Quantitative metrics resource consumption, system performance, scheduler functionality Other metrics cost, single number Needs Applications, workloads, more workloads… Benefits Configuration testing and tuning Functionality testing Comparing design alternatives Runtime estimation Systems procurement support

June 29, Grenchmark A systematic approach to testing Grid schedulers A set of metrics for comparing schedulers A set of representative applications Both real and synthetic Single tasks, MPI tasks, Java, IBIS/Satin tasks, batch tasks, filters/streaming tasks, chains of tasks, DAGs, general graphs,...(?) Easy tools to create synthetic workloads [TODAY] Testing, testing, testing… [EVERY DAY]

June 29, Grenchmark process $./wl-gen.py wl-desc.in $./wl-submit.py out/wl-to-submit.wl TODO Sample run: 95 jobs / 411 components 95 files / 32 directories 300KB data Sample run: multi-threaded – submission delay +/- 0.01s 95 JDFs

June 29, Outline Introduction Grenchmark Using Grenchmark Customizing Grenchmark Extending Grenchmark Conclusions [done] [here]

June 29, wl-desc.in $ cat wl-desc.in # File-type: text/wl-spec# WL Unit ID Times this unit Job Type Site structure type Total # of jobs per unit Detailed Site structure Arrival time distribution Other info #IDTimesUnitTypeSiteTypeTotalSiteInfoArrivalTimeDistrOtherInfo 05ssersingle1*:?U(0,1000)- 110sserunordered2-N(500,200)NSites=2,DifferentSites=false ?1sserordered?fs3:2,fs2:1,fs1:1Poisson(500)- ?1ssersemi-ordered5fs3:2,*:1LogNorm(10,5) NSites=3,DifferentSites=true ?1fakesemi-ordered5fs3:2,*:1Weibull(2000,0.5) NSites=3,DifferentSites=true ?5wl:poisson---Poisson(500) RandomWorkload=true,NJobs=10,NComponents=2;8,ComponentSize=4;8 ?2wl:poisson---Poisson(500) RandomWorkload=false,NComponents=4-8,ComponentSize=8;16 ?1wl:poisson---Poisson(500) RandomWorkload=false,NComponents=4;8,ComponentSize=8;16,Submitter=drunner,Applic ation=Poisson-g2-gm ?2wl:poisson---Poisson(500) RandomWorkload=false,NComponents=4;8,ComponentSize=8;16,Submitter=krunner,Applic ation=findpi-gm Describes the workload to be generated; input file for wl-gen.py

June 29, Preliminary notions Job, workload, workload unit Job = Set of components (co-allocation support) [Job = one program execution / the basic scheduling unit / … ] Workload = Set of Jobs Workload Unit = Set of jobs with the same property, generated in the same step (definition useful only for workload generator) Other JDF = Job description file Inter arrival time = time between two consecutive jobs arrive in the system

June 29, Some more notions Job site type / per site description Single – run at one site Unordered – run at several sites, all unspecified Ordered – run at several site, all specified Semi-ordered – run at several site, some specified Other (?) Inter-arrival time distributions Constant, Uniform, Normal, Exponential(λ), Poisson(Mean), HExp2, HPoisson2, Weibull, LogNormal, Gamma (~)

June 29, Describing the workload: Jobs wl-desc.in ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) - Generate Times x Jobs or Times x Sets of Jobs One Job = k Components (here: 1 -- single)

June 29, Demo 1: Generating one job Single site, single instance

June 29, Demo 2: Generating n jobs of the same type 250 jobs in 100s, distribution Uniform(0,1000)

June 29, Demo 3: Generating mixes of jobs 10 single jobs, 2 instances per job (count=2) Q: what happens with the running/output sandbox? 10 unordered jobs, 5 instances at 3 sites (3 components) each Q: what about ensuring DifferentSites=true? I can do it from the submitter, but then ordered… [Jan David’s P2P simulation must include controlled wide-area/single- area mixes] 10 ordered jobs, inter-arrival time (IAT) IAT 5s 10 semi-ordered jobs, 5 instances, 2 ordered sites (1 random location), IAT LogNorm(10,3)

June 29, Describing the workload: Jobs wl-desc.in ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) - Generate Times x Jobs or Times x Sets of Jobs One Job = k Components (here: 1 -- single)

June 29, Describing the workload: Units wl-desc.in ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) - ? 1 wl:poisson unordered - - Poisson(500) \ RandomWorkload=false,NComponents=4;8,ComponentSize=8;16,\ Submitter=drunner,Application=Poisson-g2-gm Generate Times x Jobs or Times x Sets of Jobs wl: prefix – self-aware workload unit generator [Hashim’s jobs]

June 29, Demo 4: Generating one workload unit Hashim’s sample workload: Poisson-g2-gm (Globus MPICH-G2) Components: 4 and 8 Component Sizes: 8 and 16 IAT Poisson(5s) Run with drunner Total: 4 jobs, 24 components

June 29, Demo 5: Generating a full workload Hashim’s full workload, 8 times: Poisson-g2-gm (Globus MPICH-G2) and findpi-gm Components: 4 and 8 Component Sizes: 4/8 and 16 IAT Poisson(5s) Run with drunner (poisson) / krunner (pi) Submit time 1 day Generate: Submit: Total: 3200 jobs, components / 3k files, 4k dirs Timing: 30s generate / 86,400s submit (1 day, eh?) $ wl-gen.py --duration= wl-desc.in $ wl-submit.py --onefile out/wl-desc.in

June 29, Outline Introduction Grenchmark Using Grenchmark Customizing Grenchmark Extending Grenchmark Conclusions [done] [here] [done]

June 29, Customizing the workload generator Command-line parameters wl-gen.py –help … displays a comprehensive help screen wl-gen.py –o outdir … creates the workload in directory outdir wl-gen.py –d … sets the total submission duration to 50s wl-gen.py –j my-jdf … uses my-jdf to write JDFs wl-gen.py –o out-1 wl-desc.in my favourite command [seen]

June 29, Customizing the submission tool Command-line parameters wl-submit.py –help … displays a comprehensive help screen wl-submit.py –t 10 … uses 10 threads to submit wl-submit.py --nosubmit … just prints the submission commands wl-submit.py --onefile … output job submit (runner) stderr and stdout to one file each (concatenated) wl-submit.py out/wl-to-submit.wl my favourite command [seen]

June 29, Outline Introduction Grenchmark Using Grenchmark Customizing Grenchmark Extending Grenchmark Conclusions [done] [here] [done]

June 29, Extending the workload generator (1) Write your own Job Generators 1.Create your-job-generator.py in apps/ 2.Place a Generator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example: Generator=""" Name=your-job-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“ 3.Create a function generateWorkloadUnit, which takes in UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom 4.Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each 5.Use the generator in wl-desc.in by putting your-job-type in the UnitType field

June 29, Extending the workload generator (2) Write your own Unit Generators 1.Create your-job-generator.py in apps/ 2.Place a WLGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example: WLGenerator=""" Name=your-workload-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“ 3.Create a function generateWorkload, which takes in UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom 4.Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each 5.Use the generator in wl-desc.in by putting your-job-type in the UnitType field

June 29, Extending the workload generator (3) Write your own JDF Writers 1.Create your-jdf-writer.py in jdfprint/ 2.Place a JDFGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example: JDFGenerator=""" Name=your-jdf-writer-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“ 3.Create a function generateJDF, which takes in OutFileName, ListOfComponents 4.Make generateJDF write the components to the specified file, with appropriate data each 5.Use the generator by calling wl-gen.py with the –j parameter

June 29, Conclusions and future work Grenchmark easily generates diverse workloads of Grid applications KOALA developers Would you like to use Grenchmark to test KOALA? Everybody Do you have specific Grid applications types that you would like to test? Try Grenchmark!

June 29, Thank you! Questions? Remarks? Observations? All welcome! Grenchmark [10x Paulo] Alexandru IOSUP TU Delft [google: “iosup”]