Frank Xu, Ph.D. Gannon University Automated Test-Input Generation Xu, W., Ding, T., Wang, H., Xu. D., Mining Test Oracles for Test Inputs Generated from Java Bytecode, Proc. of the 37th Annual International Computer Software & Applications Conference, pp , Kyoto, Japan, July 2013 Mining Decision Trees as Test Oracles for Java Bytecode (Extended version of conference paper), Accepted by Journal of Systems and Software

About Me – Frank Xu Education Ph.D. in Software Engineering, North Dakota State University M.S. in Computer Science, Towson University B.S. in Computer Science, Minor in Math, Southeast Missouri State University Working Experience GE Transportation, present, Consultant Gannon University, present, Assistant Professor of Software Engineering, Director of Keystone Software Development Institute University VA –Wise, , Assistant Professor of Software Engineering Swanson Health Products, 2005 ~ 2007, Sr. Web Programmer Analyst Volt Information Science Inc., 2004 ~ 2005, Software Engineer (Web)

Teaching Source: Student Evaluation Report

Research Source: Google scholar:

A UTOMATED T EST -I NPUT G ENERATION Introduction Software testing Test automation Test Inputs How to Generate Test Inputs Simplifying Java Code Applying rules Empirical Study/Demo Conclusions

I NTRODUCTION

Exercise Implementing a method to solve Triangle problem

What is Triangle Problem?

What is Method? Is a function or a service to complete a task A method that determines the maximum of two numbers. A method that sorts a list of names A method that opens a file from the file system Method Invoked by a method call Returns a result to calling method (caller) Similar to a boss (caller) asking a worker (called method) to complete a task

Method square returns int that result stores Method square returns the square of y y is the parameter of method square  2003 Prentice Hall, Inc.

How to test Triangle? String getTriangleType (int a, int b, int c){ if((a<b+c) && (b<a+c) && (c<a+b)){ if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ; } else return “NotATriangle “; } String getTriangleType (int a, int b, int c){ if((a<b+c) && (b<a+c) && (c<a+b)){ if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ; } else return “NotATriangle “; }

if... else Control Flow if (condition) statement1; else statement2; if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ;

Control Flow Diagram

S UMMARY : T EST T RIANGLE S TEPS Source Code Control Flow Diagram Paths (based on coverage) assertEquals(“Isosceles ”, getTriangleType(7,7,9)) Junit Test cases assertEquals(“Isosceles ”, getTriangleType(6,6,8)) ….. Step 1 Step 2 Step 3

H OW TO A UTO -G ENERATE T ESTING I NPUTS ? assertEquals (“Isosceles ”, getTriangleType(7,7,9) assertEquals (“Isosceles ”, getTriangleType(6,6,8) ….. ?

S OLUTION Randomly generate inputs (5,7,6) for Isosceles Adjust input values a=5, b=7, a==b

R EVISIT : T RIANGLE P ROBLEM

Java is Complex Statement contains comparison and expression a <b+c (Java) Condition (a<b+c) && (b<a+c) && (c<a+b)

J AVA S IMPLER V ERSION Simplify Statement a <b+c (Java) [1] $i3=i1+i2 and [2] i0>=$i3 (Jimple) Simplify condition (a<b+c) && (b<a+c) && (c<a+b) (Java) Jimple if (a<b+c) { if (b<a+c) { if(c<a+b) … }}}

IDPath 1 2 3,,,..,,, IDabc ………… … Generate inputs Path generation Generate CFG

H OW TO G ENERATE T EST I NPUTS IDabc … ………… … IDPath 1 2 3,,,

Search an input that make predicate [5]:i0>=$i3 to true a>=b+c (NotATriangle) Challenge: backtracking $i3 to input variables Recall $i3=i1+i2 Solution:

Apply Rules for Generating Test Inputs

IDPredicateExpected Evaluation Outcomes Advising Rules 1. i0 > i1(i0 > i1) = true(i0 ↑, i1) (i0, i1 ↓ ) 1. (i0 > i1) = false(i0 ↓,b) (i0, i1 ↑ ) 1. i0 == i1(i0==i1) = true(i0 ↓ D, i1)( i0, i1 ↑ D) 1. (i0== i1)= false(i0 ↑,i1) (i0, i1 ↑ ) (i0 ↓,i1) (i0, i1 ↓ ) 1. i2 = i0 + i1i2 ↑ (i0 ↑, i1) (i0, i1 ↑ ) 1. i2 ↓ (i0 ↓, i1) (i0, i1 ↓ ) 1. i2 = i0 - i1i2 ↑ (i0 ↑, i1) (i0, i1 ↓ ) 1. i2 ↓ (i0 ↓, i1)( i0, i1 ↑ ) 1. i2 = i0 * i1 (i0>0, i1>0) i2 ↑ (i0 ↑, i1) (i0, i1 ↑ ) 1. i2 ↓ (i0 ↓, i1) (i0, i1 ↓ ) 1. i2 = i0 / i1 (i0>0, i0 > 0) i2 ↑ (i0 ↑, i1) (i0, i1 ↓ ) 1. i2 ↓ (i0- ↓ i1)( i0, i1 ↑ ).. 1. s0>s1(s0 >s1) = true(s0[k] ↑, s1) (s0, s1[l] ↓ ) 1. (s0 > s1) = false(s0[k] ↓,s2) (s0, s1[l] ↑ )

E MPIRICAL S TUDY

T HREE S TUDY S UBJECTS Line of CodeNumber of Predicates JavaJimpleJava Jimple (Allow duplications) Attributes (No duplication) Triangle Next Date Vending Machine

G OAL OF E MPIRICAL S TUDIES What is the performance of the proposed approach?

IDGoalExecution Time to Generate 100 Inputs (ms) Triangle 1 Equilateral255 2 Isosceles167 3 Scalene146 4 NotATriangle121 Next Date 1 Normal day && 1st month 96 2 Last day && 1st month Normal day Last day of a normal month Normal day in Dec Last day of a year Normal day in Feb leap year Non- leap year Incorrect days of a leap year 912 Vending Machine (Change, Dollar, Cents, Juice, Beer) 1(1,1,0,0.1)534 2(1,1,0,0,0)553 3(1,0,1,1,0)523 4(1,0,1,0,1)495 5(0,1,0,1,0)277 6(0,1,0,0,0)339 7(0,0,1,1,1)285 8(0,0,1,0,1)310

Demo Path 1: Equilateral Path 5, 7, 8: Isosceles Path 6: Scalene Path 10, 11, 12: not a triangle

Key Points Understand requirements before implementation Test your code Auto test your code if possible Auto generate test inputs

Thanks

Future Research Directions Requirements Engineering & Natural language Process Generating UML diagrams, e.g., Use case, Class diagram Validating SRS Deriving test cases from SRS Software Design & Social Networks Analysis Utilizing SSA for analyzing communication diagram, class diagram, and sequence diagram for improving the quality of the software Software Implementation & Big Data Mining repository for software quality assurance using Hadoop Software Testing & Mobile/Cloud Application Testing mobile applications and distributed applications