1. Spreadsheet Table Transformations from Examples William HarrisSumit Gulwani

2. General Problem End-users have large-scale, repetitive tasks, and don’t have the right tools to do them automatically. Transform strings [POPL ‘11] Transform layout of tables

3. A Running Example: “Quals” 3 Example input table Example output table

4. 4

5. Expressiveness Ease of Use Solution Space GUI tools programs from examples traditional programming

6. A Running Example: “Quals” 6 Input table Output table

7. Demo on Quals 7

8. Methodology 1.Study a large corpus of practical transformations 2.Identify common patterns in transformations 3.Design a language: (a)expressive enough for most transformations (b)amenable to inference

9. Outline 1.Give a language of transformations 2.Give an inference algorithm for the language 3.Demonstrate inference algorithm 4.Usage and Experiments

10. Layout Program = 10 (Filter Programs, Associative Programs)

11. Transformation Step 1: Filtering 1.Filter cells from input table to output column

12. 12 Filtering Cells to Quals Column 3 01.02.2003 09.12.2009 18.04.2003 05.07.2004 31.08.2001 06.04.2007 27.06.2008 not in row 1 not in column 1 not empty

13. Filter Program: Quals Column 3 while (in_cell = get_next_row_major_cell()) { if (row(in_cell) != 1 && col(in_cell) != 1 && text(in_cell) != “”) { map_next_row(in_cell, ); } } 13 if (guard(in_cell)) { OutColumn Filter Programs: General Form Col3

14. Transformation Step 2: Association 1.Filter cells from input table to output column 2.Map spatially associated output cells from spatially associated input cells

15. Associative Program for Col 1 Col. 3 Col. 1 Lookup input Carl 18.04.2003 15 Carl

16. Associative Program: Quals Column 1 outNbor := ( ); pre := preImage(, outNbor); inCell := (pre); map(inCell, X); Associative Programs: General Form col3Filterfilter EqRowCol3AssocFuncOut EqRowCol1AssocFuncIn X

17. Outline 1.A language of transformations 2.An inference algorithm for the language 3.Demonstration of inference algorithm 4.Usage and Experiments

18. Inferring Layout Programs Inferring Filter Programs Inferring Associative Programs

19. while (in_cell = get_next_row_major_cell()) { if (guard(in_cell)){ map_next_row(in_cell, OutColumn); } 19 Filter Programs: General Form

20. Filter Predicates Cell Features row column text Constants 1, …, COL_MAX “” Predicates FEATURE(cell) = CONST FEATURE(cell) != CONST

21. 21 Filtering Cells to Quals Column 3 01.02.2003 09.12.2009 18.04.2003 05.07.2004 31.08.2001 06.04.2007 27.06.2008 cell.row != COL_1 && cell.col != ROW_1 && cell.text != “”

22. Inference Algorithm Inferring Filter Programs Inferring Associative Programs

23. outNbor := AssocFuncOut(X); pre := preImage(filter, outNbor); inCell := AssocFuncIn(pre); map(inCell, X); Associative Programs: General Form

24. Associative Functions RowEqCol1 RowEqCol2 … RowEqColN Row1ColEq

25. Inferring an Associative Program for Col 1 Col. 3 Col. 1 Lookup input Carl 18.04.2003 25 Carl Col. 2

26. Outline 1.A language of transformations 2.An inference algorithm for the language 3.Demonstration of inference algorithm 4.Usage and Experiments

27. Experiments Examples from 50+ real-world help threads 1.From example, inferred program 2.Checked program against informal description 3.If program was good: done else: extend example, repeat

30. Conclusion Motivated the problem of inferring table- layout transformations Sketched a language that describes practical transformations over table layouts Sketched an algorithm that infers programs in the language

31. Layout Program = (Filter Programs, Associative Programs) 31 Questions?

32. Extra Slides

33. Spreadsheet Table Transformations from Examples William HarrisSumit Gulwani

34. Our Approach PP Input Output 34