New Ideas Track: Testing MapReduce-Style Programs Christoph Csallner, Leonidas Fegaras, Chengkai Li csallner@uta.edu, fegaras@uta.edu, cli@uta.edu Computer Science and Engineering Department, University of Texas at Arlington (UTA), USA MapReduce = New programming model New programming model  New bugs /* Report avg of top-3 salaries, if avg>100k */ public void reduce(String dept, Iterator<Integer> salaries) { int sum = 0; int i = 0; while (salaries.hasNext() && i<3) { sum += salaries.next(); i += 1; } emit( (i>0 && sum/i > 100000)? sum/i : -1); I just implemented this lovely piece of MapReduce code. Life is great  Q: For the same input: Will your program always return the same result? A: No! Why did nobody warn me?  Can we build tools for finding such bugs automatically? Example: MapReduce programs have to satisfy certain Correctness Conditions: Map: (key;value)* Group By Key Reduce: avg. of first 3 Input Output For example: Reduce must not rely on a particular order of input values: For each input list of values L, for each permutation P: reduce(key, L) == reduce(key, P(L)) ...;dept; salary ...; A; 250,000 ...; X; 220,000 ...; F; 220,000 ...; Z; 210,000 ...; O; 150,000 ...; T; 150,000 ...; A; 150,000 ...; A; 140,000 ...; E; 100,000 ...; S; 100,000 ...; A; 95,000 ...; C; 95,000 ... A; 250,000 X; 220,000 F; 220,000 Z; 210,000 A; 250,000 A; 150,000 A; 140,000 A; 95,000 ... 180,000 Observations: 165,000 MapReduce programs are small, have little control-flow, few execution paths Dynamic symbolic execution systematically explores execution paths, precisely Approach: O; 150,000 T; 150,000 A; 150,000 A; 140,000 Derive path condition, return value, store in indexed execution tree Index leaf nodes by length of input list SiblingPath: Triggered by input list of same length Encode potential violation of correctness condition in constraint system Encode as variables: L = (L[0], L[1], ..); P(L) = (L[p[0]], L[p[1]], ..) Assert PathCond; Assert SubstituteIndices(SiblingPath, Permutation); Assert Result ≠ SubstituteIndices(SiblingResult, Permutation); Infer input values L and permutation P with off-the-shelf constraint solver Convert solution to test case, run, confirm violation B; ... ... ... ? C; ... ... E; 100,000 S; 100,000 A; 95,000 C; 95,000 ... Values may arrive in any order, depending on network, etc. ...