1 / Platform Development Process In-house limited Automatically Measuring Code Coverage in Distributed Environment Li Fei Peng Platform QA Part, NSC

2 / Platform Development Process In-house limited Content Introduction Tools for code coverage, lcov and bullseye Executing command remotely, jsch A tool for automatically measuring code cover age Troubles in this work Improvement

3 / Platform Development Process In-house limited Introduction A complicated application which runs in a distributed environment, CUBRID Code coverage need to be measured automatically Set up environment, update and compile source cod e Run test cases Process code coverage result and upload to QA web site Linux (support SSH by default)

4 / Platform Development Process In-house limited …… Introduction SVN Server Machines Running TC QA Website Control Machine

5 / Platform Development Process In-house limited Tools for code coverage Lcov –Line coverage –Function coverage –Free –Compile: -fprofile-arcs -ftest-coverage Bullseye –Function coverage –Condition/Decision coverage –Proprietary –User command cov01 to enable/disable coverage build

6 / Platform Development Process In-house limited Tools for code coverage - lcov a graphical front-end for gcov collect gcov data for multiple source files create HTML pages containin g the source code annotated with coverage information add overview pages for easy navigation within the file struc ture

7 / Platform Development Process In-house limited Tools for code coverage - bullseye Function coverage gives you a quic k overview and condition/decision c overage gives you high precision Works with everything you can writ e in C++ and C, including system-l evel and kernel mode Short learning curve Include or exclude any portion of y our project code Run-time source code included, for custom environments Simple licensing. Merge results from distributed testi ng Integration with Microsoft Visual St udio

8 / Platform Development Process In-house limited Jsch JSch is a pure Java implementation of SSH2. JSch allows you to connect to an sshd server and us e port forwarding, X11 forwarding, file transfer, etc., a nd you can integrate its functionality into your own Ja va programs. Ant provides sshexec and scp tasks by using Jsch We use two functions: –Remote exec: ls, lcov, cubrid server start –File transfer: scp, sftp

9 / Platform Development Process In-house limited Jsch We use two functions: –Remote exec: lcov cubrid server start cov01 make install … –File transfer: scp sftp

10 / Platform Development Process In-house limited A tool for automatically measuring cod e coverage Configure environment in each machines –Install lcov, bullseye, subversion –Set up environment variables, add the bin directory of lcov and bullseye to PA TH –Checkout the source code, save password Runing process –Update source code –Set up environment, compile and Install in local machine –Compress the source code and upload the source code to other machines –Set up enviroment, uncompress, compile and install in each remote machine –Run test cases –For lcov, collect code coverage data –Copy the code coverage data from other machines to control machine –Generate the visual result of code coverage in control machine –Upload the result to QA website

11 / Platform Development Process In-house limited A tool for automatically measuring cod e coverage SSHClient.java –Encapsulate the exec and file transfer functions in this class –SSHClient.exec(String host, String user, String pa ssword, String command) –SSHClient.scpFrom(String host, String user, String password, string rfile, String lfile)

12 / Platform Development Process In-house limited A tool for automatically measuring cod e coverage CovUtil.java –Main() –Update and install: updateAndInstallCubrid() –updateAndInstallCubridLocal –updateAndInstallCubridRemote –Code coverage collectCCAndUpload() –collect bullseye data or lcov data –analyze code coverage data(genhtml, no analyze for bullseye) –Upload the result to QA website

13 / Platform Development Process In-house limited A tool for automatically measuring cod e coverage Usage Usage: java com.nhncorp.cubridqa.codecoverage.C ovUtil mode: install, sa, cc 0 20 * * 5 java -jar ~/cubrid_cov/cubrid_cov.jar install ~/cubrid_cov/cov.properties 0 20 * * 6 java -jar ~/cubrid_cov/cubrid_cov.jar sa ~/cubrid_cov/cov.properties 0 23 * * 6 java -jar ~/cubrid_cov/cubrid_cov.jar cc ~/cubrid_cov/cov.properties java -jar ~/cubrid_cov/cubrid_cov.jar

14 / Platform Development Process In-house limited A tool for automatically measuring cod e coverage Configuration file –The production installation directory –The directories of related tools –QA website –Remote machines –Other parameters: removeCubrid Bullseys or lcov Exec locally or remotely Etc.

15 / Platform Development Process In-house limited A tool for automatically measuring cod e coverage cubridBuildDir=/home/xdbms/build/RB cubridHome=/home/xdbms/CUBRID lcovHome=/home/xdbms/opt/lcov-1.7 svnPath=/home/xdbms/opt/subversion/bin/svn removeCubrid=true debug=true enable64bit=false uploadHost= uploadUser=qahome uploadPassword=password uploadPath=/home/qahome/tomcat /webapps/qaresult/qaresultfile/cubrid/code coverage sqlDir=/home/xdbms/workspace/qa_repository/scenario/sql execLocal=true execRemote=true svnUpdate=true bullseyeHome=/home/xdbms/opt/bullseye enableBullseye=true #remote.1= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn #remote.2= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn #remote.3= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn #remote.4= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn #remote.5= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn #remote.6= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn remote.7= ,xdbms,password,/home/xdbms/build/RB-8.2.0,/home/xdbms/CUBRID,/home/xdbms/opt/lcov-1.7,/home/xdbms/opt/subversion/bin/svn

16 / Platform Development Process In-house limited Troubles in this work Environement variables: execute. /etc/profile. ~/.bas h_profile before executing each command Bullseye can not enabled with lcov In order to successfully merge two coverage files, th e source files used to build them must have the sam e timestamps There is some problems in utility lcov of v1.7: v1.6 is ok An error will be thrown when using genhtml: modifyin g the source code of genhtml

17 / Platform Development Process In-house limited Improvement Analyzing the data generated by bullseye Output the visual result and upload them to Q A website Code reorganization

18 / Platform Development Process In-house limited