Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Cross-application Fan-in Analysis for Finding Application-specific Concerns Makoto Ichii Takashi Ishio Katsuro Inoue Osaka University 1 AOAsia /12/2

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Coding pattern detection Automatic detection of crosscutting concerns helps  Finding refactoring opportunities  Understanding application-specific coding rules Fung: Coding pattern detection tool [Ishio, 2008][Miyake, 2007]  Detects coding patterns including crosscutting concerns from an application using a data mining technique  Basic idea: “a crosscutting concern code frequently appears across an application” 2008/12/2 AOAsia 4 2 [Ishio, 2008] T. Ishio. H. Date, T. Miyake and K. Inoue, "Mining Coding Pattern to Detect Crosscutting Concerns in Java Programs", Proc. WCRE2008, 2008 [Miyake, 2007] T. Miyake, T. Ishio, K. Taniguchi, K. Inoue, "Towards Maintenance Support for Idiom-based Code Using Sequential Pattern Mining", Proc. AOASIA3, 2007

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Example of coding pattern Coding pattern  An ordered sequence of method calls and control statements that frequently appears in source code. Process of coding pattern detection 2008/12/2 AOAsia 4 3 … if (log.isDebugEnabled()) { log.debug(getMessage()); } … String status = getStatus(); if (log.isDebugEnabled()) { log.debug(status); } … if (log.isDebugEnabled()) { log.debug("QBK"); } … isDebugEnabled() IF getMessage() debug() END_IF … getStatus() isDebugEnabled() IF debug() END_IF … isDebugEnabled() IF debug() END_IF … 1: isDebugEnabled() 2: IF 3: debug() 4: END_IF Source codeMethod call sequenceCoding pattern parse & normalize Sequential pattern mining

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Needs for application-specific concerns Detected coding patterns include generic idioms  Idioms also frequently appear across code base  Less interesting to developers who need application- specific knowledge 2008/12/2 AOAsia 4 4 1: isDebugEnabled() 2: IF 3: debug() 4: END_IF 1: iterator() 2: hasNext() 3: LOOP 4: next() 5: hasNext() 6: END_LOOP Logging Target applicationDetected patterns Iterator idiom

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Filtering approach: cross-application fan-in analysis Key Idea  Generic idioms appear in various applications  Application-specific patterns appear in a few applications  Measure how widely a class/pattern is used across applications –“Universality” metric 2008/12/2 AOAsia 4 5 1: iterator() 2: hasNext() 3: LOOP 4: next() 5: hasNext() 6: END_LOOP 1: isDebugEnabled() 2: IF 3: debug() 4: END_IF Logging Iterator idiom Appears in only two applications Appears in almost all applications

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Approach overview Collect various applications  Including target application Analyze the use-relation between the classes in the applications Measure universality metric for each classes Filter out the patterns comprising only universally-used classes. 2008/12/2 AOAsia ………………………… 2. ………………………… 3. ………………………… 4. ………………………… 5. ………………………… … 1: iterator() 2: hasNext() 3: LOOP 4: next() 5: hasNext() 6: END_LOOP 1: isDebugEnabled() 2: IF 3: debug() 4: END_IF 1: activate 2: IF 3: deactivate 4: END_IF 1: indexOf 2: lastIndexOf 3: substring 1: contains 2: IF 3: get 4: END_IF 1: iterator() 2: hasNext() 3: LOOP 4: next() 5: hasNext() 6: END_LOOP 1: isDebugEnabled() 2: IF 3: debug() 4: END_IF 1: activate 2: IF 3: deactivate 4: END_IF 1: indexOf 2: lastIndexOf 3: substring 1: contains 2: IF 3: get 4: END_IF Target application Application collectionUse-relation between classesList of universally-used classes Detected patternsFiltered patterns

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Cross-application use-relation An extension of ordinal static use-relation analysis between classes in an application. Build a use-relation graph  Node: class  Edge: static use-relation between classes Kinds of use-relation  Inheritance, Method call, Field access, Instantiation and Variable/Parameter declaration 2008/12/2 AOAsia 4 7 Source codeUse-relation graph Warehouse Liquor WarehouseApp class Liquor { long price; String name; … } class Warehouse { … Liquor liq = new Liquor(); … }

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Cross-application use-relation Analyze use-relation between classes across application borders  Analyze intra-application use-relation  in the same way with the case of single application  If there are several copies of “used class” in different applications, create edges to all of them 2008/12/2 AOAsia 4 8 Warehouse Liquor WarehouseApp Store Liquor StoreApp Paper Shelf A copy of Liquor in WarehouseApp

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Class fan-in and application fan-in 2008/12/2 AOAsia 4 9 Warehouse Liquor WarehouseApp Store Liquor StoreApp Paper Shelf Class fan-in of a class c  The number of classes using c Application fan-in of a class c  The number of applications using c AppClassCFIAFI WAWarehouse 00 Liquor 32 SAStore 00 Shelf 11 Liquor 32 Paper 21

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University The Class Universality Metric Class universality of a class c  Represents how widely a class is used  From many classes / applications 2008/12/2 AOAsia 4 10 AppClassCFIAFI WAWarehouse 00 Liquor 32 SAStore 00 Shelf 11 Liquor (copy)32 Paper 21 Univ Frequently-used universally Frequently- used locally i c : class fan-in of c ; a c : application fan-in of c ; | C | : total number of classes; | A | : total number of applications

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University The Pattern Universality Metric Pattern universality of a pattern p  The minimum universality value of the classes whose methods are invoked in p  A universal pattern comprises only universal classes 2008/12/2 AOAsia 4 11 Coding pattern ClassUniv. Collection 0.72 Iterator 0.77 Involved classes 1: iterator() 2: hasNext() 3: LOOP 4: next() 5: hasNext() 6: END_LOOP Pattern universality = 0.72

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case studies Case Study 1  Measure class universality value of actual classes Case Study 2  Measure pattern universality value of coding patterns detected by Fung 2008/12/2 AOAsia 4 12

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case Study 1 – Overview Questions Q.1 What kind of classes have high universality? Q.2 Can universality distinguish classes widely used and classes simply frequently used? Q.3 What threshold value is good for filtering? Process  Measure class universality of classes in application collection  Investigate the result to answer the questions  The top-20 classes in the universality [Q.1]  Difference between the universality and the fan-in [Q.2]  Distribution of the universality [Q.3] Target  39 application packages (131,328 classes)  Java SE 1.5  Various OSS packages covering a broad range of domains – Eclipse (IDE), Azureus (Network client), Apache Tomcat (Network server), Freemind (Drawing tool), … 2008/12/2 AOAsia 4 13

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case Study 1 – Top 20 classes in the class universality Q.1 What kind of classes have high universality?  Fundamental / Utility classes 2008/12/2 AOAsia 4 14 Class nameUniv.CFI 1 java.lang.String ,324 2 java.lang.Object ,628 3 java.util.List ,981 4 java.lang.System ,191 5 java.lang.Class ,590 6 java.lang.Throwable ,467 7 java.util.Iterator ,191 8 java.util.ArrayList ,135 9 java.lang.Exception , java.util.Map , java.lang.Integer , java.util.Set , java.io.File , java.lang.StringBuffer , java.io.PrintStream , java.util.HashMap , java.io.IOException , java.util.Collection , java.lang.IllegalArgumentException , java.lang.Runnable ,790

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case Study 1 – Universality and fan-in High universality / Low fan-in  Classes with fundamental / utility role Low universality / High fan-in  Classes implementing crosscutting concerns in a large application Q.2 Can universality distinguish classes widely used and classes simply frequently used? 2008/12/2 AOAsia 4 15 Class Name Rank [Univ.] Rank [CFI] java.lang.Character java.util.LinkedList java.io.FileOutputStream java.lang.Comparable java.util.Stack Class Name Rank [Univ.] Rank [CFI] org.eclipse.swt...Control org.eclipse.swt.SWT org.eclipse.core…IResource org.openide.util.NbBundle 1,39824 org.openide.ErrorManager 1,49654 High universality / Low fan-in Low universality / High fan-in  Yes.

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case Study 1 – Distribution : general-purpose classes  Primitive/fundamental classes, collection utilities, … : domain-specific classes  Logging utility, networking, GUI, … 0.2-0: application-local classes 2008/12/2 AOAsia 4 16 Univ.#of ClassesPackage 1.0 – 0.92 java.lang 0.9 – – java.util, java.lang, java.io 0.7 – java.lang, java.util, java.io, java.net, java.awt 0.6 – java.util, java.lang, java.io, javax.swing, java.awt, – java.io, java.lang, javax.swing, javax.swing, java.awt, – org.eclipse.swt.widgets, javax.swing, java.util, java.awt.event, java.lang, – org.eclipse.swt.widgets, org.eclipse.swt.graphics, javax.swing, javax.management, java.awt, – 0.11,385 org.eclipse.swt.widgets, org.eclipse.swt.dnd, javax.management, org.gudy.azureus2.core3.util, org.bouncycastle.asn1, – ,233 soot.jimple.parser.node, org.apache.poi.....functions, test, soot.coffi,... Q.3 What threshold value is good for filtering?  0.2 for finding application-specific concerns  0.5 for filtering out generic concerns

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case Study 2 – Overview Question  Can the pattern universality distinguish among generic, domain-specific and application-specific patterns? Process  Categorize coding patterns according to pattern universality  1.0 – 0.5: Generic pattern  0.5 – 0.2: Domain-specific pattern  0.2 – 0.0: Application-specific pattern Target  Coding patterns  Azureus (presented in [Ishio, 2008])  Application collection  Same as Case Study /12/2 AOAsia 4 17

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Case Study 2 – Result Generic patterns (2290 patterns)  String manipulation  String.lastIndexOf() / IF / String.substring() / END_IF  Collection manipulation  List.get() / IF / List.remove() / END_IF Domain-specific patterns (79 patterns)  Collection manipulation  Map.size() / Iterator.remove() / LinkedHashMap.get() / LinkedHashMap.remove() Domain-specific? Application-specific patterns (2293 patterns)  Logging  LOOP / Thread.sleep() / Debug.printStackTrace() / END_LOOP  Synchronization  IF / AEMonitor.enter() / ArrayList.remove() / AEMonitor.exit() / END_IF 2008/12/2 AOAsia 4 18 Q. Can the pattern universality distinguish generic / domain-specific / application-specific patterns?  Almost yes.

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Discussion Universality metric can distinguish universally-used classes  Resource management classes in Eclipse/NetBeans are distinguished as application-specific  although they have large fan-in Universality metric value may depend on a set of applications  Case studies in different target are needed  E.g. industrial software systems. 2008/12/2 AOAsia 4 19

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Discussion Some domain-specific classes have higher class universality than general-purpose classes -Ideas to improve the metric  Propagate fan-in through important use-relation  E.g. inheritance  Combining other metric +Less popular generic concerns may be more interesting than famous domain-specific ones 2008/12/2 AOAsia 4 20 Class nameUniv. … 33 java.awt.Component 0.63 … 113 java.util.ListIterator 0.46 … 230 java.util.LinkedHashMap 0.36 …

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University Summary and future works Cross-application fan-in analysis for filtering coding patterns  Measures universality, or a metric that represents how widely a class/pattern is used Future work  Case studies with different applications  Refinement of the universality metric 2008/12/2 AOAsia 4 21