Ran Ettinger, IBM Research – Haifa ECOOP @ Beijing, China 16 June 2012 Program Sliding Ran Ettinger, IBM Research – Haifa ECOOP @ Beijing, China 16 June 2012

for (int i = 1; i <= N; i++) { prod *= i; } print("Sum: " + sum); int prod = 1; for (int i = 1; i <= N; i++) { prod *= i; } print("Sum: " + sum); print("Product: " + prod); int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } Programmers use slices when debugging They use slices when refactoring, too Automation can help But a slice captures a subset of functionality A complementary slice (or co-slice) will capture the remaining functionality Sliding is a recomposition transformation Replace a code fragment with “slice ; co-slice” Identify threats to behavior preservation

Slice of V={sum} int prod = 1; for (int i = 1; i <= N; i++) { Sample Run initial values N = 4 System.out.* = “” int prod = 1; for (int i = 1; i <= N; i++) { prod *= i; } print("Sum: " + sum); print("Product: " + prod); int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } final values sum = 10 final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” We know how to compute slices, automatically, but what how about the complement? Definition of a slice: The slice of a code fragment S for a set of variables V is a subset of the statements of S, say SV If S terminates, SV terminates with the same result in variables V when started on the same initial state Is it the slice of all non-V variables? void print(String message) { System.out.println(message); }

Slice of CoV={prod,System.out.*} initial values N = 4 System.out.* = “” int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } int sum = 0; int prod = 1; for (int i = 1; i <= N; i++) { sum += i; prod *= i; } print("Sum: " + sum); print("Product: " + prod); The co-slice will follow the slice, so let’s (re)use its result final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” Assume variables V hold the final value on entry to the co-slice (Incomplete) Definition of a Co-slice: The co-slice of a code fragment S for a set of variables V is a subset of the statements of S, say SCoV If S terminates, SCoV terminates with the same result in all variables outside the set V when started on the same initial state void print(String message) { System.out.println(message); }

Co-slice of V={sum} int sum = 0; for (int i = 1; i <= N; i++) { initial values N = 4 System.out.* = “” sum = 10 int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } int prod = 1; for (int i = 1; i <= N; i++) { prod *= i; } print("Sum: " + sum); print("Product: " + prod); final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” Definition of a Co-slice The co-slice of a code fragment S for a set of variables V is a subset of the statements of S, say SCoV If S terminates, SCoV terminates with the same result in all variables outside the set V when started on the same initial state in all variables except V, whose initial value is the result computed by S This kind of a slice (of final values of CoV) with reuse (of V) is a (simple) case of a “fine slice” void print(String message) { System.out.println(message); } See “Fine Slicing: Theory and Applications for Computation Extraction” [Abadi, Ettinger, and Feldman, FASE @ ETAPS 2012]

Sliding for V={sum} initial values final values of the slice System.out.* = “” final values of the slice sum = 10 N = 4 System.out.* = “” int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } int prod = 1; for (int i = 1; i <= N; i++) { prod *= i; } print("Sum: " + sum); print("Product: " + prod); final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” Sliding: A Slice-Motion Transformation Isolates the slice of S on V making it contiguous Identifies the co-slice of S on V Replaces S with a sequential composition of the slice of S on V and its co-slice Expected to preserve functionality for all variables When unable, identifies problematic variables Compensatory measures may be taken to resolve those issues

Sliding for V={prod} initial values final values of the slice System.out.* = “” final values of the slice prod = 24 N = 4 System.out.* = “” int prod = 1; for (int i = 1; i <= N; i++) { prod *= i; } int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } print("Sum: " + sum); print("Product: " + prod); final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24”

Sliding for V={sum,prod} initial values N = 4 System.out.* = “” final values of the slice sum = 10 prod = 24 N = 4 System.out.* = “” int sum = 1; int prod = 1; for (int i = 1; i <= N; i++) { sum += i; prod *= i; } print("Sum: " + sum); print("Product: " + prod); final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24” final values sum = 10 prod = 24 System.out.* = “Sum: 10 Product: 24”

In the Paper… A practical sliding algorithm Suitable for (sequential) Java Building on slicing and dependence graphs Revised mechanics to refactorings Split Loop Replace Temp with Query Separate Query from Modifier Evaluation Manually refactored Eclipse’s Java compiler No detected regression

Replace Temp with Query

Replace Temp with Query Source: org.eclipse.jdt.internal.compiler.flow.UnconditionalFlowInfo String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ int i, ceil; for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { def += "," + this.extra[0][i]; //$NON-NLS-1$ pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { def += ",..."; //$NON-NLS-1$ pot += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ Source: org.eclipse.jdt.internal.compiler.flow.UnconditionalFlowInfo this fragment is a clone of another Extract Method – to make it reusable and reuse it - would fail …for two reasons: (1) ambiguous result, in variables def and pot

Replace Temp with Query Step 1: Sliding for V={def} String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ int i, ceil; for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { def += "," + this.extra[0][i]; //$NON-NLS-1$ pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { def += ",..."; //$NON-NLS-1$ pot += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$

Replace Temp with Query Step 1: Sliding for V={def} String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ int i, ceil; for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { def += "," + this.extra[0][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { def += ",..."; //$NON-NLS-1$ Source: org.eclipse.jdt.internal.compiler.flow.UnconditionalFlowInfo.toString() tucking would do the same (if we focus on the tightest SESE, the one without the return statement) KH03 would keep the loop as is, and would move the last assignment to pot to the after part

Replace Temp with Query {... String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ int i, ceil; for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { def += "," + this.extra[0][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { def += ",..."; //$NON-NLS-1$ String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ pot += "," + this.extra[1][i]; //$NON-NLS-1$ pot += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} Step 2: Extract Method, on the slice of V={def}

Replace Temp with Query {... String def = def(); int i, ceil; String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} private int def() { String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ def += "," + this.extra[0][i]; //$NON-NLS-1$ def += ",..."; //$NON-NLS-1$ return def; Step 2: Extract Method, on the slice of V={def}

Replace Temp with Query {... String def = def(); int i, ceil; String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} private int def() { String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ def += "," + this.extra[0][i]; //$NON-NLS-1$ def += ",..."; //$NON-NLS-1$ return def; Step 3: Inline Temp

Replace Temp with Query {... int i, ceil; String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return def() + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} private int def() { String def = "FlowInfo<def:[" + this.definiteInits; //$NON-NLS-1$ def += "," + this.extra[0][i]; //$NON-NLS-1$ def += ",..."; //$NON-NLS-1$ return def; Step 3: Inline Temp

Replace Temp with Query {... int i, ceil; String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return def() + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} Next, the same refactoring for V={pot} requires no sliding

Replace Temp with Query {... return def() + pot() + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} private String pot() { int i, ceil; String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return pot; Next, the same refactoring for V={pot} requires no sliding

Replace Temp with Query {... return def() + pot() + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} private String pot() { int i, ceil; String pot = "], pot:[" + this.potentialInits; //$NON-NLS-1$ for (i = 0, ceil = this.extra[0].length > 3 ? 3 : this.extra[0].length; i < ceil; i++) { pot += "," + this.extra[1][i]; //$NON-NLS-1$ } if (ceil < this.extra[0].length) { pot += ",..."; //$NON-NLS-1$ return pot; Finally, let’s replace the clone too

Replace Temp with Query ...for (...) { def += "," + this.extra[0][i]; //$NON-NLS-1$ pot += "," + this.extra[1][i]; //$NON-NLS-1$ nullS += "," + this.extra[2][i] //$NON-NLS-1$ + this.extra[3][i] + this.extra[4][i] + this.extra[5][i]; } if (ceil < this.extra[0].length) { def += ",..."; //$NON-NLS-1$ pot += ",..."; //$NON-NLS-1$ nullS += ",..."; //$NON-NLS-1$ return def + pot + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + nullS + "]>"; //$NON-NLS-1$ else { if (this.extra == null) { return ...; //$NON-NLS-1$ return def() + pot() + ", no null info>"; //$NON-NLS-1$ ...} Second reason Extract Method would have failed: this is not an exact clone Finally, let’s replace the clone too

Replace Temp with Query ...for (...) { nullS += "," + this.extra[2][i] //$NON-NLS-1$ + this.extra[3][i] + this.extra[4][i] + this.extra[5][i]; } if (ceil < this.extra[0].length) { nullS += ",..."; //$NON-NLS-1$ return def() + pot() + "], reachable:" + ((this.tagBits & UNREACHABLE) == 0) //$NON-NLS-1$ + nullS + "]>"; //$NON-NLS-1$ else { if (this.extra == null) { return ...; //$NON-NLS-1$ + ", no null info>"; //$NON-NLS-1$ ...} The loop and conditional are duplicated now 3 times instead of 2 (so 6 statements instead of 4) The exclusive 3 statements in the computation of def are no longer duplicated (3 instead of 6) The exclusive 3 statements in the computation of pot are no longer duplicated (3 instead of 6) Total: added 2 statements and removed 6!

Separate Query from Modifier

Separate Query from Modifier Source: org.eclipse.jdt.internal.compiler.codegen.ConstantPool public int literalIndex(int key) { int index; if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } if ((index = this.intCache.putIfAbsent(key, this.currentIndex)) < 0) { this.currentIndex++; if ((index = -index) > 0xFFFF){ someSideEffects(); // one (long) statement here someMoreSideEffects(index, key); // 11 statements here return index;

Separate Query from Modifier public int literalIndex(int key) { int index; if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } index = this.intCache.getValue(key, this.currentIndex); this.intCache.putIfAbsent(key, this.currentIndex); if (index < 0) { this.currentIndex++; index = -index; if (index > 0xFFFF){ someSideEffects(); // one (long) statement here someMoreSideEffects(index, key); // 11 statements here return index; 2 manual updates: SQfM and an inlined equivalent

Separate Query from Modifier public int literalIndex(int key) { int index; if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } index = this.intCache.getValue(key, this.currentIndex); this.intCache.putIfAbsent(key, this.currentIndex); if (index < 0) { this.currentIndex++; index = -index; if (index > 0xFFFF){ someSideEffects(); // one (long) statement here someMoreSideEffects(index, key); // 11 statements here return index; The return could be inside the scope for sliding, but I prefer to exclude it so the co-slice will be directly extracted as the modifier (without the return)

Separate Query from Modifier int index; if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } index = this.intCache.getValue(key, this.currentIndex); if (index < 0) { index = -index; if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } index = this.intCache.getValue(key, this.currentIndex); this.intCache.putIfAbsent(key, this.currentIndex); if (index < 0) { this.currentIndex++; if (index > 0xFFFF){ someSideEffects(); // one (long) statement here someMoreSideEffects(index, key); // 11 statements here Sample input: this.intCache = null key = 40 this.currentIndex = 6 Modified values after slice with no compensation: this.intCache = new object {(40,6)} index = 6 Modified or relevant values after co-slice with no compensation: this.currentIndex = 6 (instead of 7) index = 6 (luckily recomputed to the same value) Modified or relevant values after slice with compensation, renaming this.intCache in the slice, copying its initial value: localIntCache = a new object {(40,6)} Modified values after co-slice with compensation, localizing index in the co-slice: this.intCache = a 2nd new object {(40,6)} this.currentIndex = 7 localIndex = 6

int index; if ( == null) { = new IntegerCache(INT_INITIAL_SIZE); } index = .getValue(key, this.currentIndex); if (index < 0) { index = -index; IntegerCache intCache = this.intCache; intCache if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } index = this.intCache.getValue(key, this.currentIndex); this.intCache.putIfAbsent(key, this.currentIndex); if (index < 0) { this.currentIndex++; if (index > 0xFFFF){ someSideEffects(); // one (long) statement here someMoreSideEffects(index, key); // 11 statements here

int index; if ( == null) { = new IntegerCache(INT_INITIAL_SIZE); } index = .getValue(key, this.currentIndex); if (index < 0) { index = -index; IntegerCache intCache = this.intCache; intCache int index1; index1 if (this.intCache == null) { this.intCache = new IntegerCache(INT_INITIAL_SIZE); } = this.intCache.getValue(key, this.currentIndex); this.intCache.putIfAbsent(key, this.currentIndex); if ( < 0) { this.currentIndex++; if (index > 0xFFFF){ someSideEffects(); // one (long) statement here someMoreSideEffects(index, key); // 11 statements here

Evaluation A manual experiment Sliding for Replace Temp with Query Refactoring the well-tested code of the open-source Java compiler in Eclipse Update the code as needed in preparation for the refactoring Sliding for Replace Temp with Query Loops with more than one result Sliding for Separate Query from Modifier Non-void methods with side effects (detected using WALA’s ModRef analysis) Update the code as needed in preparation for the refactoring Split statements that declare more than one variable Split called methods if the result is needed in the slice and the side effects in the co-slice Separate side effects from expressions in conditionals

Evaluation 55 refactorings 77 manual preparatory changes 23 successful cases of sliding for Replace Temp with Query 28 successful cases of sliding for Separate Query from Modifier A single problematic case returning the value a field (see paper) 3 remaining cases would work (depending on successful SQfM of a problematic polymorphic call) 77 manual preparatory changes

Evaluation Code size: 6 to 97 statements Average: 23.5 Slice size: 1 to 85 statements Average: 10.4 Co-slice size: 1 to 82 statements Average: 20 (duplicated 6.9)

Evaluation No compensation needed in 44% of the cases (7 RTwQ and 17 SQfM) In all remaining cases variable renaming and simple initial-value backup of up to 3 variables was sufficient Reuse of slice result 132 final-value uses 107 non-final uses 46 non-final uses remain (in 22 co-slices)

Related Work: Statements as Input Tucking [Lakhotia & Deprez, IST98] No reuse of slice results in the complement Rejected when a variable is modified on both sides Semantics preserving procedure extraction [Komondoor & Horwitz, POPL00] Rejected when any duplication is needed Effective, automatic procedure extraction [Komondoor & Horwitz, IWPC03] If a statements cannot be moved it is added to the extracted code Only predicates and jumps may be duplicated

Related Work: Variables as Input Block-based slice extraction [Maruyama, SSR01] Extract the slice of a single variable Reuse the slice result, but incorrectly for non-final uses too Identification of extract method refactoring opportunities [Tsantalis & Chatzigeorgiou, CSMR09,JSS11] Solved Maruyama’s correctness issue through rules for identifying and rejecting problematic cases No duplication of a statement that modifies the state of an object

A Limitation of Sliding for Variables No way to extract the code for “computing and printing sum” in the following: int prod = 1; for (int i = 1; i <= N; i++) { prod *= i; } print("Sum: " + sum); print("Product: " + prod); int sum = 0; for (int i = 1; i <= N; i++) { sum += i; } Need support for the extraction of intermediate values See “Fine Slicing” [Abadi et al., FASE12] for a general approach

Conclusion Sliding is a new way of recomposing programs automatically Focuses on final-value slices of sets of variables Useful in refactoring Testing, verification, and compiler optimizations too? Practical algorithm based on program dependence graphs Some topics for further work: interprocedural and amorphous sliding (avoid need for manual code changes, remove duplication) automation of the refactorings sliding for intermediate values too sliding for any selection of statements reverse sliding to merge matching loops accuracy vs. speed investigations (e.g. of pointer analyses)

Thanks! Join (or follow) our ongoing WALA-based refactoring tool implementation effort: https://wala.svn.sourceforge.net/svnroot/wala/incubator/com.ibm.wala.refactoring Past and present contributors: Alex Libov, Shay Menaia, Dima Rabkin, Vlad Shumlin, Eli Kfir, Daniel Lemel, Moshe Zemah Special thanks to Steve Fink