Many to many, too many performance tests Christoph Baumer, Biometrical Practice BIOP, Basel, Switzerland PhUSE 2011 CS03

1/10/2010Many to many, too many performance tests2 In software engineering, performance testing is testing that is performed, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Wikipedia

1/10/2010Many to many, too many performance tests3 In software engineering, performance testing is testing that is performed, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Wikipedia

1/10/2010Many to many, too many performance tests4 What is needed for performance testing in a clinical environment? - Transparent approach - Modular and flexible setting of testing environments - Simulation of real world scenarios

1/10/2010Many to many, too many performance tests5 Available options 1. Running a single program in a batch job. 2. Running a program multiple times. 3. Running programs in different orders

1/10/2010Many to many, too many performance tests6 Many to many Example – merging WHO ATC text with drugname WHO.THG medprodATCcodeofficial 5C02ABY 47R03CBY 373G03AAN J07BCN D06BBY D06BBY S01GXY S01GXY WHO.MP MedprodDrugname ALOMIDE BONDIL BONDIL EVISTA EVISTA HUMALOG IMDUR IMDUR WHO.ATC ATCcodeATCtxt S01FBSympathomimetics excl. antiglaucoma preparations S01GDECONGESTANTS AND ANTIALLERGICS S01GASympathomimetics used as decongestants S01GXOther antiallergics S01HLOCAL ANESTHETICS S01HALocal anesthetics S01JDIAGNOSTIC AGENTS S01JAColouring agents

1/10/2010Many to many, too many performance tests7 Many to many Programs Program 1 & 2: SQL join Program 3 & 4: Point option Program 5:Formats Program 6: Sorting and merging

1/10/2010Many to many, too many performance tests8 Program 1: %let description=sql inner join; proc sql; create table atc1 as select atc.atctxt, mp.drugname from thg inner join atc on thg.atccode eq atc.atccode inner join mp on thg.medprod eq mp.medprod; quit;

1/10/2010Many to many, too many performance tests9 Program 2: %let description=sql join with where clause; proc sql; create table atc2 as select atc.atctxt, mp.drugname from thg, atc, mp where thg.atccode eq atc.atccode and thg.medprod eq mp.medprod; quit;

1/10/2010Many to many, too many performance tests10 Many to many, too many performance tests Program 3: %let description=point loops with mp inside; data atc3; set thg; do k = 1 to nobs_atc; set atc(rename = (atccode = _atccode_)) nobs=nobs_atc point=k; if atccode eq _atccode_ then do; do l = 1 to nobs_mp; set mp(rename = (medprod = _medprod_)) nobs=nobs_mp point=l; if medprod = _medprod_ then output; end; keep atctxt drugname; run;

1/10/2010Many to many, too many performance tests11 Many to many, too many performance tests Program 4: %let description=point loops with atc inside; data at4; set thg; do k = 1 to nobs_mp; set mp(rename = (medprod = _medprod_)) nobs=nobs_mp point=k; if medprod = _medprod_ then do; do l = 1 to nobs_mp; set atc(rename = (atccode = _atccode_)) nobs=nobs_atc point=l; if atccode eq _atccode_ then output; end; keep atctxt drugname; run;

1/10/2010Many to many, too many performance tests12 Many to many, too many performance tests Program 5: %let description=Using formats; proc sql; create table fmt_mp as select medprod as start, drugname as label, 'mp' as fmtname, 'n' as type from mp; create table fmt_atc as select atccode as start, atctxt as label, 'atc' as fmtname, 'c' as type from atc; quit; proc format cntlin=fmt_mp; run; proc format cntlin=fmt_atc; run; data atc5; set thg; acttxt = put(atccode,atc.); drugname = put(medprod,mp.); keep atctxt drugname; run;

1/10/2010Many to many, too many performance tests13 Many to many, too many performance tests Program 6: %let description=sorting and merging; proc sort data=atc; by atccode; run; proc sort data=thg; by atccode; run; data atc0; merge atc thg; by atccode; keep atctxt medprod; run; proc sort data=mp; by medprod; run; proc sort data=atc0; by medprod; run; data atc0; merge atc0 mp; by medprod; keep atctxt drugname; run;

1/10/2010Many to many, too many performance tests14 Many to many, too many performance tests Base 2: libname who "D:\many_to_many\lib"; data thg; set who.thg; where official eq 'N'; run; proc sql; create table mp as select * from who.mp where medprod in (select medprod from thg); create table atc as select * from who.atc where atccode in (select atccode from thg); quit; Base 1: libname who "D:\many_to_many\lib"; data mp; set who.mp; run; data thg; set who.thg; run; data atc; set who.atc; run; Base 3: libname who "D:\many_to_many\lib"; data thg; set who.thg; if _n_ le 2000; run; proc sql; create table mp as select * from who.mp where medprod in (select medprod from thg); create table atc as select * from who.atc where atccode in (select atccode from thg); quit; Base conditionNumber of observations in result dataset BASE BASE BASE32000

1/10/2010Many to many, too many performance tests15 Many to many, too many performance tests PROG1 PROG2 PROG3 … BASE1 BASE2 … PROGx BASEx Main Folder is read and program names are stored %inc(BASE1) %inc(PROG1) %inc(BASE1) %inc(PROG2) %inc(BASE2) %inc(PROG1) %inc(BASE2) %inc(PROG1) %inc(BASE1) %inc(PROG3) %inc(BASE2) %inc(PROG3) Programs with all combinations are created

1/10/2010Many to many, too many performance tests16 Many to many, too many performance tests %inc(BASE1) %inc(PROG1.SAS) %inc(BASE1) %inc(PROG1) Running a single program in a batch job

1/10/2010Many to many, too many performance tests17 Many to many, too many performance tests %inc(BASE1) %inc(PROG1.SAS) %inc(PROG2.SAS) %inc(BASE1) %inc(PROG1.SAS) %inc(PROG1.SAS) Running a program multiple times Running programs in different orders

1/10/2010Many to many, too many performance tests18 Many to many, too many performance tests descriptiondurationbasenameprognamerepeatorder sql with inner join 9.089base2.sasprog1.sas1prog1.sas, prog2.sas sql join with where clause 7.418base2.sasprog2.sas2prog1.sas, prog2.sas sql with inner join base2.sasprog1.sas1prog1.sas, prog5.sas Using formats14.117base2.sasprog5.sas2prog1.sas, prog5.sas sql with inner join base2.sasprog1.sas1prog1.sas, prog6.sas sorting and merging base2.sasprog6.sas2prog1.sas, prog6.sas sql join with where clause base2.sasprog2.sas1prog2.sas, prog1.sas sql with inner join 8.73base2.sasprog1.sas2prog2.sas, prog1.sas sql join with where clause base2.sasprog2.sas1prog2.sas, prog5.sas Using formats17.022base2.sasprog5.sas2prog2.sas, prog5.sas Results dataset

1/10/2010Many to many, too many performance tests19 Many to many, too many performance tests &repeats: Specifies the number of repeats of a single program within a single run. This will add a program multiple times to a certain file / batch job. &n_runs: Specifies how many times each batch job is executed. &multiple: Specifies, if two programs run within the same batch job. Program options

1/10/2010Many to many, too many performance tests20 Many to many, too many performance tests Used files: prog1 – prog6, base3 %perf(n_runs=10); Compare programs with BASE3 as basis DescriptionFilenameMean duration (seconds) sql inner joinprog1.sas sql join with where clauseprog2.sas point loops with mp insideprog3.sas point loops with atc insideprog4.sas Using formatsprog5.sas Sorting and mergingprog6.sas0.0515

1/10/2010Many to many, too many performance tests21 Many to many, too many performance tests Used files: prog1,prog2, prog5, prog6, base2 %perf(n_runs=5,repeats=3); Compare programs with BASE2, running each program 3 times Mean duration (seconds) DescriptionFirst repeatSecond repeatThird repeat sql with inner join sql join with where clause Using formats Sorting and merging

1/10/2010Many to many, too many performance tests22 Many to many, too many performance tests Used files: prog1,prog2, prog5, prog6, base1 %perf(n_runs=5,repeats=3); Compare programs with BASE1, running each program 3 times Mean duration (seconds) DescriptionFirst repeatSecond repeatThird repeat sql inner join sql join with where clause Using formats Sorting and merging

1/10/2010Many to many, too many performance tests23 Many to many, too many performance tests Used files: prog1,prog2, prog5, prog6, base2 %perf(n_runs=10,multiple=YES); Compare programs with BASE2, running all combinations of programs Mean duration in seconds Previous Description sql inner join sql join with where clause Using formats Sorting and merging Program run at first place sql inner joinNA sql join with where clause NA Using formats NA Sorting and merging NA

Why? 1/10/2010Many to many, too many performance tests24 Many to many, too many performance tests Prepared for the final run Improve your programming Gain understanding of SAS

Thank you!

Questions? 1/10/2010Biop Presentation Title26 Many to many, too many performance tests