A Tool for Constructing Syntax-Directed Editors Yung-Shen Chang and Nai-Wei Lin Department of Computer Science and Information Engineering National Chung-Cheng University Taiwan, R.O.C.

2 Outline Introduction Introduction A generator for incremental parsers A generator for incremental parsers A simple interface to incremental parsers A simple interface to incremental parsers Conclusions and future work Conclusions and future work

3 Typical Program Development Cycle Program editing and program compilation are separated. Program editing and program compilation are separated. It is more efficient if we can do program editing and program compilation at the same time. It is more efficient if we can do program editing and program compilation at the same time. Edit Compile Found BugsEdit Compile … Found Bugs

4 Syntax-Directed Editors Syntax-directed editors have the ability to perform program editing and program compilation at the same time. Syntax-directed editors have the ability to perform program editing and program compilation at the same time. Syntax-directed editors have become an integral feature for modern integrated development environment. Syntax-directed editors have become an integral feature for modern integrated development environment.

5 Incremental Parsing The main capability of syntax-directed editors is incremental parsing. The main capability of syntax-directed editors is incremental parsing. Incremental parsing has the ability to only parse the modified portion of a program. Incremental parsing has the ability to only parse the modified portion of a program. This ability allows us to interleave program editing and program compilation. This ability allows us to interleave program editing and program compilation.

6 Contributions A generator for incremental parsers that is based on Bison. A generator for incremental parsers that is based on Bison. A simple interface to incremental parsers to facilitate the integration of editors and incremental parsers to form syntax- directed editors. A simple interface to incremental parsers to facilitate the integration of editors and incremental parsers to form syntax- directed editors.

7 Bison Grammar definition (sample.y) Parser (sample.tab.c) Bison

8 Outline Introduction Introduction A generator for incremental parsers A generator for incremental parsers A simple interface to incremental parsers A simple interface to incremental parsers Conclusions Conclusions

9 Incremental Parsing Algorithms The state matching incremental parsing algorithm (Larchêveque) The state matching incremental parsing algorithm (Larchêveque) –no need to change Bison parsing table The sentential-form incremental parsing algorithm (Wagner) The sentential-form incremental parsing algorithm (Wagner) –need to change Bison parsing table

10 Threaded Tree Parsing Use threaded tree data structure to represent the parse tree Use threaded tree data structure to represent the parse tree Threaded tree is a combination of parse tree and parse stack Threaded tree is a combination of parse tree and parse stack All shift/reduce actions are operated on the threaded tree All shift/reduce actions are operated on the threaded tree

11 Threaded Parse Tree Each node in the threaded tree has the following fields Each node in the threaded tree has the following fields –symbol –children –state –threading

12 Shift Action Shift symbol M at state S Shift symbol M at state S 1.construct a node N (M, S) 2.set N ’s threading to TOS 3.set TOS (Top of Stack) as N N1N1 N (M, S) TOS

13 Reduce Action Reduce symbol M at State S with k children Reduce symbol M at State S with k children 1.construct a node N (M, S) 2.collect k nodes from TOS 3.construct connections between N and its children 4.set N ’s threading to leftmost node of its children 5.set TOS as N N1N1 N(M,S) s1s1 r2r2 TOS → M → sr

14 An Example S1 f2f6F3 G4 g6 H7 h9 BOS Input= fghf Action: Shift f2 Shift g6 Reduce G4 Shift h9 Reduce H7 Reduce F3 Shift f6 Reduce S1 TOS

15 Incremental Threaded Tree Parsing Split previous parse tree into three parts x y z Split previous parse tree into three parts x y z x and z are unmodified parts x and z are unmodified parts y is the modified part y is the modified part Let y’ be the modifying content Let y’ be the modifying content xyz x, z =unmodified y =modified

16 Incremental Threaded Tree Parsing 1. Finding initial configuration (x) 2. Parse exhaustively (y’) 3. Finding optimal grafting point from candidates (z) 4. Perform node grafting

17 Finding Candidates A grafting point must be a common ancestor of y’ and y A grafting point must be a common ancestor of y’ and y We start searching from nearest common ancestor (NCA) We start searching from nearest common ancestor (NCA)

18 Proper Grafting Points A candidate for a grafting point is an NCA n such that A candidate for a grafting point is an NCA n such that –terminal successor of n = current lookahead –symbol of n = symbol of the TOS –predecessor of n = predecessor of the TOS

19 An Example S1S1 BOS 0 f2f2 g5g5 G4G4 h9h9 H7H7 F3F3 f6f6 g8g8 G4G4 TOS NCA lookahead Grafting Point F3F3 = deleted node x : g 5 y : g 8 z : h 9, f 6

20 Incremental Parser Modules node Parsing table reader tree incparser parser manage read Determine what to do Perform action (shift/reduce) read … node = module = file Perform grafting if find suitable grafting point input Bison Parsing Table

21 Outline Introduction Introduction A generator for incremental parsers A generator for incremental parsers A simple interface to incremental parsers A simple interface to incremental parsers Conclusions Conclusions

22 Interface to Incremental Parser Editor calls incparser to get service from incremental parser Editor calls incparser to get service from incremental parser Editor passes parameters begin, end, and delta Editor passes parameters begin, end, and delta incparser will store the parsing result in variable err incparser will store the parsing result in variable err voidincparser (POS begin, POS end, const string& delta, ERR& err);

23 Flow of an Incremental Parsing Session incparser interface editor Write delta Return parse result Read input from temporary file Call interface Wrapper call incparser Temporary file

24 Proper Timing to Trigger Incremental Parsing By timer By timer By each key press By each key press By editing model By editing model

25 Editing Model Editing is a sequence of key presses Editing is a sequence of key presses Classify key presses into two groups Classify key presses into two groups –modikeys – which will cause content change –non-modikeys – which won’t cause content change Editing causes a state change between pressing modikeys and non-modikeys Editing causes a state change between pressing modikeys and non-modikeys Editor should remember BEGIN and END position when a state change occurs Editor should remember BEGIN and END position when a state change occurs

26 Special Keys BS (Backspace)/DEL are keys in modikeys that need special treatments BS (Backspace)/DEL are keys in modikeys that need special treatments BS might change the BEGIN position BS might change the BEGIN position DEL might change the END position DEL might change the END position Editor should perform appropriate action when user pressing BS/DEL Editor should perform appropriate action when user pressing BS/DEL We use a counter to maintain how many modikeys are pressed (except BS/DEL) We use a counter to maintain how many modikeys are pressed (except BS/DEL)

27 An Example Assume that the cursor is at the position between “ ma ”, “ in ”, the editor is at non- modikey state Assume that the cursor is at the position between “ ma ”, “ in ”, the editor is at non- modikey state User performs the following modifications User performs the following modifications –press BS two times –press DEL two times –key in “ foo_function ” –key in any non-modikeys

28 An Example (cont.) Initial situation Initial situation –cursor at (1, 7) –BEGIN and END is N/A Pressing BS two times Pressing BS two times –cursor at (1, 5) –BEGIN = (1, 5) –END = (1, 8) }04 return 0;03 printf(“Hello, World\n”);02int main (void) {01 }04 return 0;03 printf(“Hello, World\n”);02 int in (void) {01

29 An Example (cont.) Press DEL two times Press DEL two times –cursor at (1, 5) –BEGIN = (1, 5) –END = (1, 10) }04 return 0;03 printf(“Hello, World\n”);02int in (void) {01}04 return 0;03 printf(“Hello, World\n”);02int (void) {01

30 An Example (cont.) Key in “ foo_function ” Key in “ foo_function ” –cursor at (1, 17) –BEGIN = (1, 5) –END = (1, 10) –DELTA = “ foo_function ” Press any non-modikeys Call incparser by Press any non-modikeys Call incparser by –BEGIN = (1, 5) –END = (1, 10) –DELTA = “ foo_function ” }04 return 0;03 printf(“Hello, World\n”);02int (void) {01}04 return 0;03 printf(“Hello, World\n”);02int foo_function (void) {01

31 Outline Introduction Introduction A generator for incremental parsers A generator for incremental parsers A simple interface to incremental parsers A simple interface to incremental parsers Conclusions Conclusions

32 Conclusions We developed a generator for incremental parsers based on Bison We developed a generator for incremental parsers based on Bison We introduced a simple interface to incremental parsers that facilitate integration of an incremental parser and an editor based on editing model We introduced a simple interface to incremental parsers that facilitate integration of an incremental parser and an editor based on editing model

33 Future Work A generator for incremental lexers A generator for incremental lexers A generator for incremental semantics analyzers A generator for incremental semantics analyzers A generator for syntax-directed editors A generator for syntax-directed editors