1 Extending the Eclipse Debug Perspective and JUnit to Support Testing and Debugging of Domain-Specific Languages August 16, 2005 Hui Wu and Jeff Gray Department of Computer and Information Sciences University of Alabama at Birmingham Magic City Java User ’s Group

2 Categories of End-Users Admin Assistants Businessman Auto Factory Worker Scientist Albert Einstein Spreadsheet Business Query Systems Modeling Language DSL for Physics

3 Problems Computer errors cost economy millions of dollars each year  $60 billion per year in US [CNN05, TR02] The dangers of end-user programming [Har05]  End-users often lack knowledge of software development principles  Inadequate testing and debugging processes  No testing tools support for end-users (especially for DSL users)

4 Motivation of Proposed Research Domain Experts program at DSL level DSL translated into General Purpose Language (GPL) Domain Experts deal with DSL Integrated Development Environment (IDE) Editor Compiler Visualizer Debugge r Domain Experts deal with GPL Test Engine Profiler

5 Mismatch Between Abstraction Levels …. commands : ( c:command cs:commands | ) ; command : ( RIGHT { fileio.print("//move right"); fileio.print("x=x+1;"); fileio.print("time=time+1;"); fileio.print(" "); } |LEFT { fileio.print("//move left"); fileio.print("x=x-1;"); fileio.print("time=time+1;"); fileio.print(" "); … …. public final void commands() throws RecognitionException, TokenStreamException { try { // for error handling { switch ( LA(1)) { case CALL: case INIT: case SET: case PRINT: { command(); commands(); break; } case END: { break; } default: { throw new NoViableAltException(LT(1), getFilename()); … public final void function_name() throws RecognitionException, TokenStreamException { try { // for error handling { switch ( LA(1)) { case RIGHT: { match(RIGHT); fileio.print("//move right"); fileio.print("move_right();"); fileio.print(" "); break; } case LEFT: { match(LEFT); sllinenumber=dsllinenumber+1; fileio.print("//move left"); fileio.print("move_left();"); fileio.print(" "); break; } … ANTLR Grammar Generated Java Parser Code

6 Difficulties with Building DSL Testing Tools Debuggers and Profilers are difficult to build because they depend heavily on the underlying operating system’s capabilities and lower-level native code functionality Manual construction of the test tools for each new DSL can be time-consuming, expensive, and error-prone

7 Background Eclipse Debugger Platform Eclipse Unit Test Platform Design Maintenance System (DMS)

8 Research Directions Imperative DSLs Declarative DSLs Hybrid DSLs Debuggers Unit Test Engines Profilers construction of the same software tool (e.g., debugging) to different categories of DSLs vertical direction corresponds to the vector representing the various testing tools that will be applied to DSLs

9 Research Goal A Matrix of DSL Testing Tools generalized approach to produce a software product line of testing tools for DSLs

10 A Horizontal Direction: Debuggers for Different Types of DSLs Imperative DSL: Centered around assignment expressions or control flow statements  Robot Language … 17 Down: 18 position(+0,-1) 19 Down: F_end 20M_end 21 Init position(0,0) 22 Call left 23 Call down 24 Call knight 25 Set position(5,6) 26 Call up 27 Call right 28 Print position …

11 A Horizontal Direction: Debuggers for Different Types of DSLs Declarative DSL: declares the relationship between inputs and outputs.  Feature Definition Language (FDL) 1 Car : all (Carbody, Transmission, Engine, Horsepower, opt(pullsTrailer)) 2 Transmission : oneof (automatic, manual) 3 Engine : moreof (electric, gasoline) 4 Horsepower : oneof (lowPower, mediumPower, highPower) 5 include pullsTrailer 6 pullsTrailer requires highPower

12 A Horizontal Direction: Debuggers for Different Types of DSLs Hybrid DSL: embedded GPL code within the DSL description … 80 functionbodys 81 :(functionbody functionbodys 82 | 83 ) 84 ; functionbody 87 :(VARIABLES LPAREN op1:OP func_num1 :NUMBER COMMA op2:OP func_num2:NUMBER RPAREN 88 { funcall="functionbody"; 89 dsllinenumber=dsllinenumber+1; 90 fileio.print("x=x"+op1.getText()+func_num1.getText()+";"); 91 gplbeginline=fileio.getLinenumber(); 92 fileio.print("y=y"+op2.getText()+func_num2.getText()+";"); 93 fileio.print("time=time+1;"); 94 gplendline=fileio.getLinenumber(); 95 filemap.print("mapping.add(newMap("+dsllinenumber+",\"Robot.java\"," 96 +gplbeginline+","+gplendline+","+"\""+funcname+"\"" 97 +","+"\""+funcall+"\""+"));"); 98 } 99 ) 100 ; …

13 Plug-In Based Software Development Syntax-Directed Translation Weaving Aspects into DSL Grammars Approaches Used in This Project Software Design Patterns Automated Software Engineering Eclipse Model-View-Controller Adapter Pattern AspectG ANTLR 6 after(int commandname): 7 call(void antlr.Parser.match(int)) 8 && args(commandname) 9 { match(commandname); } 10 pointcut count_dsllinenumber(): 11 call (void P.command()); 12 after(): count_dsllinenumber(){ 13 { dsllinenumber=dsllinenumber+1;}

14 DSL Debugger Framework (DDF)

15 Processes of DSL Debugger Generation DSL grammar is defined using ANTLR Notation DSL is the input of the Lexer and Parser generated by ANTLR DSL translation process Robot DSL corresponding GPL code generated in Java Additional Mapping code generated in Java Corresponding GPL and Mapping Code are inputs of Mapping component jdb communicates with mapping component Eclipse Debugging perspective communicates with mapping component

16 Debugging: A Crosscutting Grammar Concern dsllinenumber=dsllinenumber+1; gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber(); filemap.print(" mapping.add(new Map("+dsllinenumber+", \"Robot.java\","+gplbeginline+","+gplendline+"));"); … command :( RIGHT { fileio.print("//move right"); fileio.print("x=x+1;"); fileio.print("time=time+1;"); fileio.print(" "); } |LEFT { fileio.print("//move left"); fileio.print("x=x-1;"); fileio.print("time=time+1;"); fileio.print(" "); } | … dsllinenumber=dsllinenumber+1; gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber(); filemap.print(" mapping.add(new Map("+dsllinenumber+", \"Robot.java\","+gplbeginline+","+gplendline+"));"); Base Grammar Duplicate Debugging Aspect Code What if this line changes? Change here Change ……

17 Weaving into DSL Grammars Using AspectG DSL Grammar ANTLR Parlanse Functions Parser’ Lexer’ With Debugging Aspects Weaved in DSL Code Debugging Mapping Code GPL In Java DMS DSL Grammar’ With Debugging Aspect Weaved In Debugging Aspect Specification In AspectG

18 pointcut productions(): within(command.*); pointcut count_gpllinenumber(): within(command.*) && match (fileio.print("time=time+1;")); before(): productions() { dsllinenumber=dsllinenumber+1;} after(): count_gpllinenumber() {gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber();} after(): productions() {filemap.print(" mapping.add(new Map("+ dsllinenumber+", \"Robot.java\","+gplbeginline+ ","+gplendline+"));"); } … command :( RIGHT { dsllinenumber=dsllinenumber+1; fileio.print("//move right"); fileio.print("x=x+1;"); fileio.print("time=time+1;"); gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber(); fileio.print(" "); filemap.print(" mapping.add(new Map("+ dsllinenumber+", \"Robot.java\","+gplbeginline+ ","+gplendline+"));"); } … Pointcut Model for AspectG ANTLR Grammar AspectG

19 DSL Debugger Perspective in Eclipse

20 Declarative DSL Debugger

21 Demonstration

22 DSL Unit Test Framework (DUTF) Similar to the DSL Debugging Framework architecture and approaches Complement to the DSL Debugging Framework, identify the existence of program errors To monitor the performance of DSLs at run time, similar architecture and approaches can be applied to DSL Profiler Framework (DPF) to generate DSL profilers

23 DSL Unit Test Perspective in Eclipse

24 DSL Unit Test Perspective in Eclipse

25 Demonstration

26 Summary End-user developers (e.g., scientist, accountant, and statistician) are in large number Software failures caused by program errors pose a great threat to economy Testing and Debugging on GPLs have been investigated for decades, but testing and debugging DSLs has been neglected. Several publication reviews, paper citations, and source code requests indicate the proposed research is in the right direction.

27 Thank you for coming! Questions? Summary of URLs referenced in talk  DSL Debugger Framework  ANTLR  AspectJ  DMS

28 Backup Slides

29 Experimental Validation of Proposed Work Various types of DSLs serve as test cases  Robot Language (imperative DSL)  FDL (declarative DSL)  ANTLR grammar (hybrid DSL) Qualitative Metrics-How well the generated testing tools behave  To what degree of success can the DSL unit test engine detect errors in DSL program units?  To what degree can the DSL debugger help end-users to locate and fix the detected errors?

30 Related Work ASF+SDF  Does not generate test engines and profilers  ToolBus Integrated Debugging Environment (TIDE), 4 tedious steps to implement debugger for a new language  AspectAsf: weave debugging aspects into all potential program locations. Khepera  Support optimization code debugger  Store transformation information in a database LISA  Debugger, test engine, and profiler are not the target language tools, LISA can be used as the front-end of our framework [HPM+05].

31 Related Work SmartTools  Base on Java and XML technologies  Debugger, test engine, and profiler are not target language tools JTS  Complicated mechanism (e.g., Language extension)  Dose not provide rich transformation ability JastAdd  Aspect weaving in Compiler construction Aspect-Oriented Compiler  Aspect weaving in Compiler construction

32 First Approach: Weaving at GPL Level DSL Grammar ANTLR Parser Lexer In Java Debugging Aspect Specification In AspectJ AspectJ Compiler Parser’ Lexer’ With Debugging Aspects Weaved in DSL Code Debugging Mapping Code GPL In Java

33 First Approach: Weaving at GPL Level … 6 after(int commandname): 7 call(void antlr.Parser.match(int)) 8 && args(commandname) 9 { match(commandname); } 10 pointcut count_dsllinenumber(): 11 call (void P.command()); 12 after(): count_dsllinenumber(){ 13 { dsllinenumber=dsllinenumber+1;} … … try { // for error handling { switch ( LA(1)) { case RIGHT: case LEFT: case UP: case DOWN: case INIT: case SET: case PRINT: { command(); commands(); break; } case END: { System.out.println(" "); break; } default: { throw new NoViableAltException(LT(1), getFilename()); … Generated Parser code by ANTLR Debugging Aspect in AspectJ

34 First Approach: Weaving at GPL Level Disadvantages: The lack of mature aspect weavers for many languages (e.g., Object Pascal, C, or Ada) Requires the developer of the DSL to have detailed knowledge of the code generator within ANTLR in order to construct the appropriate pointcuts In some cases, the generated code is unreadable by a human

35 Second Approach: Weaving into DSL Grammars DSL Grammar ANTLR Debugging Aspect Specification In Parlanse Function Parser’ Lexer’ With Debugging Aspects Weaved in DSL Code Debugging Mapping Code GPL In Java DMS DSL Grammar’ With Debugging Aspect Weaved In

36 Second Approach: Weaving into DSL Grammars Transform actual DSL Grammar  Unique contribution of this research Language independence. 12 Steps to Weave Debugging Aspects into DSL Grammar Level Using a Program Transformation Engine (DMS) 1. Specify ANTLR grammar specification 2. Specify Java semantic actions using DMS regular expression 3. Generate ANTLR Parser 4. Generate abstract syntax tree with ANTLR_ACTION nodes 5. Search ANTLR_ACTION nodes from the generated AST 6. Retrieve ANTLR_ACTION nodes and store them in a hash map 7. Retrieve associated string expression from each ANTLR_ACTION node 8. Modify the regular Java parser by changing the starting production 9. Parse the associated string expressions as regular Java statement lists 10.Transform the statement lists using the ASTInterface API 11.Regenerate the ANTLR_ACTION nodes with debugging aspects weaved in 12.Output the complete ANTLR AST (with modified action nodes) … command :( RIGHT { dsllinenumber=dsllinenumber+1; fileio.print("//move right"); fileio.print("x=x+1;"); fileio.print("time=time+1;"); gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber(); fileio.print(" "); filemap.print(" mapping.add(new Map("+dsllinenumber+", \"Robot.java\","+gplbeginline+","+gplendline+"));"); } |LEFT { dsllinenumber=dsllinenumber+1; fileio.print("//move left"); fileio.print("x=x-1;"); fileio.print("time=time+1;"); gplbeginline=fileio.getLinenumber(); gplendline=fileio.getLinenumber() fileio.print(" "); filemap.print(" mapping.add(new Map("+dsllinenumber+", \"Robot.java\","+gplbeginline+","+gplendline+"));"); } …