1. Many to many, too many performance tests Christoph Baumer, Biometrical Practice BIOP, Basel, Switzerland PhUSE 2011 CS03

2. 1/10/2010Many to many, too many performance tests2 In software engineering, performance testing is testing that is performed, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Wikipedia

3. 1/10/2010Many to many, too many performance tests3 In software engineering, performance testing is testing that is performed, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Wikipedia

4. 1/10/2010Many to many, too many performance tests4 What is needed for performance testing in a clinical environment? - Transparent approach - Modular and flexible setting of testing environments - Simulation of real world scenarios

5. 1/10/2010Many to many, too many performance tests5 Available options 1. Running a single program in a batch job. 2. Running a program multiple times. 3. Running programs in different orders

6. 1/10/2010Many to many, too many performance tests6 Many to many Example – merging WHO ATC text with drugname WHO.THG medprodATCcodeofficial 5C02ABY 47R03CBY 373G03AAN 768726J07BCN 768727D06BBY 768728D06BBY 768729S01GXY 768730S01GXY WHO.MP MedprodDrugname 768730ALOMIDE 768731BONDIL 768732BONDIL 768733EVISTA 768734EVISTA 768735HUMALOG 768736IMDUR 768737IMDUR WHO.ATC ATCcodeATCtxt S01FBSympathomimetics excl. antiglaucoma preparations S01GDECONGESTANTS AND ANTIALLERGICS S01GASympathomimetics used as decongestants S01GXOther antiallergics S01HLOCAL ANESTHETICS S01HALocal anesthetics S01JDIAGNOSTIC AGENTS S01JAColouring agents

7. 1/10/2010Many to many, too many performance tests7 Many to many Programs Program 1 & 2: SQL join Program 3 & 4: Point option Program 5:Formats Program 6: Sorting and merging

8. 1/10/2010Many to many, too many performance tests8 Program 1: %let description=sql inner join; proc sql; create table atc1 as select atc.atctxt, mp.drugname from thg inner join atc on thg.atccode eq atc.atccode inner join mp on thg.medprod eq mp.medprod; quit;

9. 1/10/2010Many to many, too many performance tests9 Program 2: %let description=sql join with where clause; proc sql; create table atc2 as select atc.atctxt, mp.drugname from thg, atc, mp where thg.atccode eq atc.atccode and thg.medprod eq mp.medprod; quit;

10. 1/10/2010Many to many, too many performance tests10 Many to many, too many performance tests Program 3: %let description=point loops with mp inside; data atc3; set thg; do k = 1 to nobs_atc; set atc(rename = (atccode = _atccode_)) nobs=nobs_atc point=k; if atccode eq _atccode_ then do; do l = 1 to nobs_mp; set mp(rename = (medprod = _medprod_)) nobs=nobs_mp point=l; if medprod = _medprod_ then output; end; keep atctxt drugname; run;

11. 1/10/2010Many to many, too many performance tests11 Many to many, too many performance tests Program 4: %let description=point loops with atc inside; data at4; set thg; do k = 1 to nobs_mp; set mp(rename = (medprod = _medprod_)) nobs=nobs_mp point=k; if medprod = _medprod_ then do; do l = 1 to nobs_mp; set atc(rename = (atccode = _atccode_)) nobs=nobs_atc point=l; if atccode eq _atccode_ then output; end; keep atctxt drugname; run;

12. 1/10/2010Many to many, too many performance tests12 Many to many, too many performance tests Program 5: %let description=Using formats; proc sql; create table fmt_mp as select medprod as start, drugname as label, 'mp' as fmtname, 'n' as type from mp; create table fmt_atc as select atccode as start, atctxt as label, 'atc' as fmtname, 'c' as type from atc; quit; proc format cntlin=fmt_mp; run; proc format cntlin=fmt_atc; run; data atc5; set thg; acttxt = put(atccode,atc.); drugname = put(medprod,mp.); keep atctxt drugname; run;

13. 1/10/2010Many to many, too many performance tests13 Many to many, too many performance tests Program 6: %let description=sorting and merging; proc sort data=atc; by atccode; run; proc sort data=thg; by atccode; run; data atc0; merge atc thg; by atccode; keep atctxt medprod; run; proc sort data=mp; by medprod; run; proc sort data=atc0; by medprod; run; data atc0; merge atc0 mp; by medprod; keep atctxt drugname; run;

14. 1/10/2010Many to many, too many performance tests14 Many to many, too many performance tests Base 2: libname who "D:\many_to_many\lib"; data thg; set who.thg; where official eq 'N'; run; proc sql; create table mp as select * from who.mp where medprod in (select medprod from thg); create table atc as select * from who.atc where atccode in (select atccode from thg); quit; Base 1: libname who "D:\many_to_many\lib"; data mp; set who.mp; run; data thg; set who.thg; run; data atc; set who.atc; run; Base 3: libname who "D:\many_to_many\lib"; data thg; set who.thg; if _n_ le 2000; run; proc sql; create table mp as select * from who.mp where medprod in (select medprod from thg); create table atc as select * from who.atc where atccode in (select atccode from thg); quit; Base conditionNumber of observations in result dataset BASE11826677 BASE2494285 BASE32000

15. 1/10/2010Many to many, too many performance tests15 Many to many, too many performance tests PROG1 PROG2 PROG3 … BASE1 BASE2 … PROGx BASEx Main Folder is read and program names are stored %inc(BASE1) %inc(PROG1) %inc(BASE1) %inc(PROG2) %inc(BASE2) %inc(PROG1) %inc(BASE2) %inc(PROG1) %inc(BASE1) %inc(PROG3) %inc(BASE2) %inc(PROG3) Programs with all combinations are created

16. 1/10/2010Many to many, too many performance tests16 Many to many, too many performance tests %inc(BASE1) %inc(PROG1.SAS) %inc(BASE1) %inc(PROG1) Running a single program in a batch job

17. 1/10/2010Many to many, too many performance tests17 Many to many, too many performance tests %inc(BASE1) %inc(PROG1.SAS) %inc(PROG2.SAS) %inc(BASE1) %inc(PROG1.SAS) %inc(PROG1.SAS) Running a program multiple times Running programs in different orders

18. 1/10/2010Many to many, too many performance tests18 Many to many, too many performance tests descriptiondurationbasenameprognamerepeatorder sql with inner join 9.089base2.sasprog1.sas1prog1.sas, prog2.sas sql join with where clause 7.418base2.sasprog2.sas2prog1.sas, prog2.sas sql with inner join 10.525base2.sasprog1.sas1prog1.sas, prog5.sas Using formats14.117base2.sasprog5.sas2prog1.sas, prog5.sas sql with inner join 10.401base2.sasprog1.sas1prog1.sas, prog6.sas sorting and merging 50.677base2.sasprog6.sas2prog1.sas, prog6.sas sql join with where clause 10.823base2.sasprog2.sas1prog2.sas, prog1.sas sql with inner join 8.73base2.sasprog1.sas2prog2.sas, prog1.sas sql join with where clause 11.182base2.sasprog2.sas1prog2.sas, prog5.sas Using formats17.022base2.sasprog5.sas2prog2.sas, prog5.sas Results dataset

19. 1/10/2010Many to many, too many performance tests19 Many to many, too many performance tests &repeats: Specifies the number of repeats of a single program within a single run. This will add a program multiple times to a certain file / batch job. &n_runs: Specifies how many times each batch job is executed. &multiple: Specifies, if two programs run within the same batch job. Program options

20. 1/10/2010Many to many, too many performance tests20 Many to many, too many performance tests Used files: prog1 – prog6, base3 %perf(n_runs=10); Compare programs with BASE3 as basis DescriptionFilenameMean duration (seconds) sql inner joinprog1.sas0.0171 sql join with where clauseprog2.sas0.0203 point loops with mp insideprog3.sas1.7721 point loops with atc insideprog4.sas2.0143 Using formatsprog5.sas0.0782 Sorting and mergingprog6.sas0.0515

21. 1/10/2010Many to many, too many performance tests21 Many to many, too many performance tests Used files: prog1,prog2, prog5, prog6, base2 %perf(n_runs=5,repeats=3); Compare programs with BASE2, running each program 3 times Mean duration (seconds) DescriptionFirst repeatSecond repeatThird repeat sql with inner join12.46769.948813.3372 sql join with where clause13.616620.890226.3024 Using formats18.486824.08820.0874 Sorting and merging45.079627.177227.1866

22. 1/10/2010Many to many, too many performance tests22 Many to many, too many performance tests Used files: prog1,prog2, prog5, prog6, base1 %perf(n_runs=5,repeats=3); Compare programs with BASE1, running each program 3 times Mean duration (seconds) DescriptionFirst repeatSecond repeatThird repeat sql inner join53.7402112.0068101.496 sql join with where clause67.5392123.6426170.0498 Using formats50.8864139.606289.4168 Sorting and merging230.2746101.711495.9858

23. 1/10/2010Many to many, too many performance tests23 Many to many, too many performance tests Used files: prog1,prog2, prog5, prog6, base2 %perf(n_runs=10,multiple=YES); Compare programs with BASE2, running all combinations of programs Mean duration in seconds Previous Description sql inner join sql join with where clause Using formats Sorting and merging Program run at first place sql inner joinNA13.636711.47388.336411.3405 sql join with where clause12.9232NA12.75449.526310.9484 Using formats19.79920.4817NA17.071117.2770 Sorting and merging51.215651.796450.1929NA44.8809

24. Why? 1/10/2010Many to many, too many performance tests24 Many to many, too many performance tests Prepared for the final run Improve your programming Gain understanding of SAS

25. Thank you!

26. Questions? 1/10/2010Biop Presentation Title26 Many to many, too many performance tests