Building Documentation Generators Arie van Deursen Tobias Kuipers CWI, The Netherlands

Documentation Maintenance outsourcing increases demand for documentation Software company asked us whether we could generate documentation Our reverse engineering results can be intuitively presented as (a particular form of) documentation

Documentation Challenges Should be available at different levels of abstraction User must be able to move between different abstractions Abstraction level should be meaningful to user Needs to be consistent with source code

Tool Architecture System Sources Automated Fact Extractor Repository Manual Fact Extraction Hand-written Documentation Formatter Documentation

Source Code Analysis Lexical Analysis (regular expressions) –CALL [A-Z]+ Syntactical Analysis –* CALL FOO –CALL FOO USING... –CALL \n FOO

Combine lexical and syntactical syntax “*” ~[\n]* “\n” -> LAYOUT [A-Z]+ -> Id ~[\.]+ “.” -> Statement “CALL” Id “.” -> Statement Statement+ -> Program priorities “CALL” Id “.” -> Statement > ~[\.]+ “.” -> Statement

Example Analysis MOVE 1 TO A. * This is some comment CALL * Strange place for comment FOO. PERFORM BAR.

Example Analysis MOVE 1 TO A. * This is some comment CALL * Strange place for comment FOO. PERFORM BAR.

Example Analysis MOVE 1 TO A. * This is some comment CALL * Strange place for comment FOO. PERFORM BAR.

Example Analysis MOVE 1 TO A. * This is some comment CALL * Strange place for comment FOO. PERFORM BAR.

Island Grammars Detailed production rules for relevant constructs Catch-all productions with lower priority Minimal set of productions for overall program structure Used for rapid/specific parser development

Repository and Presentation Standard tools –Relational database –Hypertext browser Different levels of abstraction –Preformulated queries presented as hypertext to maintenance programmer –Reengineer can formulate own queries on database Demo

Contributions Island Grammars for fact extraction –Integration of hand-written facts/documentation Explicit Repository/Data model Flexible formatting Integration of visual/textual presentation Applied in several commercial projects