1. Deep Learning for Program Repair
Aditya Kanade
Indian Institute of Science
Dagstuhl Seminar on Automated Program Repair, January 2017

2. A Probabilistic Perspective on Program Repair
Specification = A set of examples
= { (x, y) | x is a faulty program and y is its fixed version }
Represent the faulty and fixed programs as sequences of tokens
x = x1, x2, …, xn and y = y1, y2, …, yk
Learn a conditional probability distribution P(Y = y | X = x)
Repair procedure: To fix an unseen program x, generate a y such that
y = argmax P(Y = y’ | X = x)

3. Seq2seq Neural Net with Attention Mechanism
Attention mechanism to focus
on diff. parts of input sequence
With fixed-size context vector

4. Neural Network Architecture
Encoder
Decoder

5. DeepFix: Fixing Common Programming Errors
Common programming errors: Unlike logical errors, these are not specific to a programming task at hand, but relate to the overall syntax and structure of the programming language
Analogous to grammatical errors in natural languages.
Both experienced developers and novice students make them
Detected by compilers as syntactic errors, but are neither accurately localized nor fixed by the compilers

6. Example Input faulty program with attention weights
Program as repaired by DeepFix

7. The Iterative Repair Strategy of DeepFix
An end-to-end deep learning solution where the neural net
learns to both localize and fix the faults.

8. Experimental Setup
Identifiers (variable and method names) are canonicalized
Resulting vocabulary size is 129, tokens embedded in 50-dimensional vectors
Seq2seq w/ attention network implemented in TensorFlow
Both encoder and decoder are 4 stacked GRU layers of 300 cells in each layer
Dropout at the rate of 0.2 on non-recurrent connections
Training configuration
Trained using ~150K examples created synthetically
Stochastic gradient descent using Adam optimizer with mini-batch of 128
Gradients clipped at [-1,1]
Training time of 1-2 hours per epoch
Select the model with peak validation performance over 20 epochs

9. Results
Evaluated on 6971 faulty C programs submitted to 93 programming tasks in an introductory programming tasks
Fixed 27% programs completely such that the programs compile w/o errors
Fixed additional 19% programs partially
Evaluated on 9230 C programs with seeded faults
Fixed 56% programs completely
Fixed additional 17% programs partially
Localization accuracy on seeded faults: Top-1 = 79%, Top-5 = 89%
Types of errors fixed: missing delimiters, extraneous symbols, swapped symbols, missing variable declarations, etc.

10. Some Plots PCA projects of vector-representations of
correct (circles) and faulty (triangles) programs
Distribution of fixed programs by tasks

11. Lightweight Machine Learning

12. Statistical Correlation to Search for Likely Fixes
Given a test suite and a potential faulty location, how to search for expressions that are likely to appear in the fixed version?
Use symbolic execution to obtain desired expression values
Enumerate a set of expressions and their values on all executions
Rank expressions using statistical correlation with the desired values
Used for synthesis of repair hints in MintHint [ICSE’14]

13. Clustering Programs for Large-Scale Feedback Generation on Student Code? ? ? ? ? ? ? ? ? ?
Clustering ? ? ? ? ? ? ? ?
by solution
strategy ? ? ? ? ? ? ? ? ?
Instructor certifies
one submission from
each cluster as correct
Student programs ?  ?  ? ?     ? 
Verified feedback ?    ?  ?   ?   ? ?
Used in CoderAssist [FSE’16] to generate feedback on student programs

14. Acknowledgements
DeepFix: Fixing common C language errors by deep learning
Rahul Gupta, Soham Pal, Aditya Kanade, Shirish Shevade
AAAI’17
Semi-supervised verified feedback generation
Shalini Kaleeswaran, Anirudh Santhiar, Aditya Kanade, Sumit Gulwani
FSE’16
MintHint: Automated synthesis of repair hints
Shalini Kaleeswaran, Varun Tulsian, Aditya Kanade, Alessandro Orso
ICSE’14
Some images taken from the Deep Learning book by Goodfellow, Bengio and Courville

15. Discussion Points
Improving performance of DeepFix, e.g., using a copying mechanism?
Fixing more challenging programming errors?
Handling larger programs? Learning better dependences?
General-purpose deep nets versus special-purpose deep nets designed for program text?
How to obtain high-quality training data? Mutation strategies?