2. Tempus Software Maintenance and Evolution JMSE-SM&E Unit 7: Reverse Engineering Prof. Mohammad A. Mikki Gaza, Palestine, 5.8.2014

3. Joint Master in Software Engineering “Software Maintenance and Evolution “ Course Unit 7 Reverse Engineering Software Maintenance and Evolution 2

4. Joint Master in Software Engineering Objectives/ILOs  Define and get an overview of reverse engineering  Understand the relationship between reverse engineering and re-engineering  Understand the issues and complete a reverse engineering project  Be able to analyse reverse engineering options  Evaluate different reverse engineering strategies  Differentiate between different levels of analysis in reverse engineering  Use static and dynamic analysis techniques of reverse engineering  Differenciate between analysis of source code vs. binaries. Software Maintenance and Evolution 3

5. Joint Master in Software Engineering List of Topics  Overview  Reverse Engineering Process  Reverse Engineering Methodology  System Level Design  Data reverse engineering  Source code vs. binaries  Static and Dynamic Analysis  Reverse engineering tools  Example on reverse engineering Software Maintenance and Evolution 4

6. Joint Master in Software Engineering Reverse Engineering Definition Reverse engineering is the process of analysing a system to identify its components and their interrelationships with the objective to create representations of the system in another form or at a higher level of abstraction. Software Maintenance and Evolution 5

7. Joint Master in Software Engineering  Reverse engineering often precedes re-engineering.  It may be part of a re-engineering process but it may also be used to re-specify a system for re-implementation.  Reverse engineering is sometimes done as an independent activity. Reverse Engineering Concepts Software Maintenance and Evolution 6

8. Joint Master in Software Engineering  Reverse engineering can be done at many levels.  Reverse engineering belongs to Software Maintenance.  Reverse engineering is a systematic methodology for analyzing the design of an existing system, either as an approach to study the design or as a prerequisite for re- design. Reverse Engineering Concepts (Cont.) Software Maintenance and Evolution 7

9. Joint Master in Software Engineering  Determine how a product works.  Learn the ideas and technology that were used in developing the product.  Identify the system's components and their interrelationships.  Create representations of the system in another form or at a higher level of abstraction. Reverse Engineering Objectives Software Maintenance and Evolution 8

10. Joint Master in Software Engineering  You have the following problem: - By accident, you delete the a source code file -How to recover it?  Answer: - Use reverse Engineering Why Do We Need Reverse Engineering? Software Maintenance and Evolution 9

11. Joint Master in Software Engineering  Discovery of abstraction Abstract System Built System Reverse Engineering Why Do We Need Reverse Engineering? (Cont.) Software Maintenance and Evolution 10

12. Joint Master in Software Engineering  Savings in maintenance cost  Improvement of system’s quality  Competitive advantages  Facilitation of software reuse Benefits of Reverse Engineering Software Maintenance and Evolution 11

13. Joint Master in Software Engineering Relation With Forward Engineering  Reverse engineering is the opposite of forward engineering.  Forward engineering is when a system is designed and built to do its specified objectives using basic already designed components or subsystems.  In forward engineering, the system is already built and the engineers have to find out how it is designed, e.g., which components are used to build it. Software Maintenance and Evolution 12

14. Joint Master in Software Engineering Abstract System Built System System to be built Reverse Engineering or Abstraction Forward Engineering or Implementation Relation With Forward Engineering (Cont.) Software Maintenance and Evolution 13

15. Joint Master in Software Engineering Reverse Engineering Phases  Phase 1: Collecting information by studying the existing system -Through tools such as parsers, debuggers, profilers, and event recorders  Phase 2: Abstracting information -Finding out the the high level components which compose the system Software Maintenance and Evolution 14

16. Joint Master in Software Engineering Reverse Engineering Process [1] Software Maintenance and Evolution 15

17. Joint Master in Software Engineering  Re-documentation and/or document generation  Recovery of design approach and design details at any level of abstraction  Identifying reusable components and components that need restructuring  Recovering business rules  Understanding high-level system description. Reverse Engineering Methodology [4] Software Maintenance and Evolution 16

18. Joint Master in Software Engineering 1. Investigation, Prediction and Hypothesis 2. Concrete Experience: Function & Form 3. Design Models 4. Design Analysis 5. Parametric Redesign 6. Adaptive Redesign7. Original Redesign Reverse Engineering Modeling & Analysis Redesign Reverse Engineering Methodology (Cont.) [2] Software Maintenance and Evolution Figure: Reverse Engineering Methodology [3]  Steps in the figure will be explained in following slides 17

19. Joint Master in Software Engineering  Develop black box model  Use / Experience product  List assumed working principles  Perform economic feasibility of redesign  State process description or activity diagram 1.Investigation, Prediction and Hypothesis [2] Software Maintenance and Evolution 18

20. Joint Master in Software Engineering  Plan and execute product disassembly  Group defined systems and components together  Experiment with product components  Develop free body diagrams  Identify function sharing and compatibility  Transform to engineering specification and metrics 2. Concrete Experience: Function and Form [2] Software Maintenance and Evolution 19

21. Joint Master in Software Engineering  Identify actual physical principles  Constantly consider the customer  Create engineering models and metric ranges  Alternatively or concurrently build prototype to test parameters 3. Design Models [2] Software Maintenance and Evolution 20

22. Joint Master in Software Engineering  Calibrate model  Create engineering analysis, simulation or optimization  Create experiment and testing procedures 4. Design Analysis [2] Software Maintenance and Evolution 21

23. Joint Master in Software Engineering  Optimize design parameters  Perform sensitivity analysis and tolerance design  Build and test prototype 5. Parametric Redesign [2] Software Maintenance and Evolution 22

24. Joint Master in Software Engineering  Recommend new subsystems  Search for inventive solutions  Analyze force flows and component combinations  Build and test prototype 6. Adaptive Redesign [2] Software Maintenance and Evolution 23

25. Joint Master in Software Engineering  Develop new functional structure  Choose alternatives  Verify design concepts  Build and test prototype 7. Original Redesign [2] Software Maintenance and Evolution 24

26. Joint Master in Software Engineering To handle the complexity of system design, the system is described at different levels of abstraction (top-down design approach): 1. System Level Design: Defines system partition into sub-systems and their interface. 2. Sub-system level design (recursive task): For each sub-system, define its sub-systems and their interface Abstraction Levels of System Design Software Maintenance and Evolution 25

27. Joint Master in Software Engineering System Level Design  It is defining components, modules, parts, interfaces, structure, and architecture of the system to satisfy its requirements.  System level design is required by reverse engineering Software Maintenance and Evolution 26

28. Joint Master in Software Engineering System Structure (Architecture) System is composed of:  Components -Parts or modules of the system. -They are the subsystem components. -They define the system at component-level  Connections -between components -Specify how components interface and communicate Software Maintenance and Evolution 27

29. Joint Master in Software Engineering  System is modeled as a black box.  System gets input from the environment.  System produces output to the environment. System Interaction with Environment Input Output System Software Maintenance and Evolution 28

30. Joint Master in Software Engineering System Level Design Example: TV Convert Signal To Sound At Desired Level + Convert Frequency to Video Channel Power TV Signal User Choices (Volume, Frequency,) Heat & Noise Sound + Video Status Indications (Volume, Frequency) Software Maintenance and Evolution 29

31. Joint Master in Software Engineering Data Reverse Engineering  Data reverse engineering focuses on data and data- relationships for two types of data: -data structures within programs -databases Software Maintenance and Evolution 30

32. physical schema: -Data -Schema catalog -Code -documentation logical schema conceptual schema Analysis -domain expert -- developer -- reengineer Abstraction - reengineer extension migration wrapping integration distribution... Data Reverse Engineering [3] - keys Optimizations -... - OO model (objects, associations, inheritance,...) Software Maintenance and Evolution31

33. Joint Master in Software Engineering Data Reverse Engineering Exampe  Convert text data files into relational databases (DBs) e.g., converting the following flat text data file into a relational DB table. Column name Width First name 25 Last name 25 Id 9 Street25 City20 Country10 Birth Date 8 Software Maintenance and Evolution 32

34. Source Code vs. Binaries for SE  What is source code: -It is the code of a program written in high level programming language. -Contains variable declarations, instructions, functions, loops, and other statements that tell the program how to function. -Comments could be added to explain code sections.  What is binary code: -It is the simplest form of computer code or programming data. -It is represented entirely by a binary system of digits consisting of a string of consecutive zeros and ones. -Binary code is often associated with machine code Software Maintenance and Evolution33

35. Source Code vs. Binaries for SE (Cont.) [3] Using source code  better form of representation  not always possible  result depends on the parser (notable differencies) Using binaries  faster information collection (e.g. Java byte code)  legality issues Software Maintenance and Evolution34

36. Joint Master in Software Engineering  Recovery of lost source code  Migration of applications to a new hardware platform  Translation of code written in obsolete languages not supported by compiler tools nowadays  Determination of the existence of viruses or malicious code in the program  Recovery of someone else's source code (e.g., to determine an algorithm) Usage of Binaries (reverse engineering, decompilation, disassembly) [3] Software Maintenance and Evolution 35

37. Joint Master in Software Engineering public class MyTest { // This is a silly program. public static void main(String[] args) { int myInt1=1; int myInt2=2; for (int i=1;i<10;i++) { for (int j=2;j<8;j++) myInt1++; myInt2=myInt2+myInt1; } System.out.println("myInt1 is " + myInt1 + " and myInt2 is" + myInt2); } -> Compiled with Sun’s javac compiler and decompiled with DJ Java Decompiler, let’s see what we got (see next slide): A Decompilation Example [3] Software Maintenance and Evolution 36

38. Joint Master in Software Engineering import java.io.PrintStream; public class MyTest { public MyTest() { } public static void main(String args[]) { int i = 1; int j = 2; for(int k = 1; k < 10; k++) { for(int l = 2; l < 8; l++) i++; j += i; } System.out.println("myInt1 is " + i + " and myInt2 is" + j); } A Decompilation Example (Cont.) [3] Software Maintenance and Evolution 37

39. Joint Master in Software Engineering Static and Dynamic Models [3]  Static Model: Finding out the static structure, architecture - code (using a parser) - Documents -Interviews  Dynamic Model: Finding out the run-time behavior of software -Debugger -Profiler -Source code instrumentation Software Maintenance and Evolution 38

40. Joint Master in Software Engineering Static Model Visualization [3]  Class diagrams  Hierarchical graphs Software Maintenance and Evolution 39

41. Joint Master in Software Engineering  Scenarios - (sequence diagrams)  State diagrams - (hierarchical) graphs Dynamic Models Visualization [3] Software Maintenance and Evolution 40

42. Joint Master in Software Engineering  Abstracting the high-level components (like subsystems)  The process types - Automatic abstraction Using the structure of the language Using measurements - Manual abstraction Abstracting the Static Model [3] Software Maintenance and Evolution 41

43.  CodeCrawler: a reverse engineering tool that combines metrics and graphs to visualize OO systems  http://www.iam.unibe.ch/~lanza/codecrawler/codecrawler.html Example: Using CodeCrowler Reverse Engineering Tool [3] Software Maintenance and Evolution42

44. Joint Master in Software Engineering  Finding behavior patterns, repeating sequences of events - E.g. initializing a dialogue  Using static abstractions - E.g. representing interactions between high-level software elements in sequence diagrams  Dynamic information is combined with the high-level static model Abstracting the Dynamic Model [3] Software Maintenance and Evolution 43

45. Merging static and dynamic information to a single view Dynamic and static views - connections and correspondencies between the views need to be defined + both static and dynamic abstractions can be built + static and dynamic views are separated also in forward engineering: support for re- engineering and roun-trip engineering + more informatin can be viewed - Directly illustrates connections between static and dynamic info - Ensuring the quality of the view -polymorfism (OO) may cause confusion - building abstractions becomes combersome and/or requires trade offs: bahavioral patterns subsystems - sequential information is difficult to merge to a static view - the more informatin a view contains, the less readable it gets ! Merging Static and Dynamic Models [3] Software Maintenance and Evolution44

46. Joint Master in Software Engineering Analyzing the Static Model [3]  Syntax, type checking, interfaces  Control and data flow analysis  Structure analysis  Slicing and dicing (different ways to partition the software)  Measuring the complexity  Navigation Software Maintenance and Evolution 45

47. Joint Master in Software Engineering  Object creation and related dependencies  Dynamic binding, polymorphism  Method calls  Looking for dead code/reachability analysis  Memory management  Performance and related problems  Concurrency Analysing the Dynamic Model [3] Software Maintenance and Evolution 46

48. Joint Master in Software Engineering Reverse Engineering Tools [3]  Tools supporting creation of high-level models  Tools supporting metrics  Forward & reverse engineering - Re-engineering & round-trip-engineering &testing  Other tools - Parser generators - Design pattern recognition Software Maintenance and Evolution 47

49. Joint Master in Software Engineering RE Tool Examples [3]  Rigi (University of Victoria, Canada) - http://www.rigi.csc.uvic.ca/ - A research prototype that represents an open and public domain reverse engineering tool - User programmable - Analysis for: C, C++, COBOL, PL/AS, LaTeX  SNIFF+ (TakeFive Software) - A software development environment that also provides reverse engineering capabilities Software Maintenance and Evolution 48

50. Joint Master in Software Engineering RE Tool Examples [3]  McCabe’s Visual Reengineering Toolset and Visual Quality Toolset - Various views - Software metrics (ccomplexity and structuredness) shown as specific colors on the views  Logiscope (CS Verilog) - Reverse engineering, code testing, static and dynamic testing, metrics - Analysis for: C, C++, Java, ADA  ESW (Viasoft Inc.) - Forward and reverse engineering (maintenance), metrics, testing Software Maintenance and Evolution 49

51. Joint Master in Software Engineering RE Tool Examples [3]  Refine (Reasoning Systems Inc.) -An open and programmable tool that works in the Refinery environment tools for generating source code parsing and conversion tools -Features for analyzing and re-engineering code analysis for: Ada, C, Cobol  Imagix4D (Imagix Corp.) -http://www.powersoftware.com/english/im/index.html -A closed tool that provides a large set of built-in functionalities -Several views (also 3D) -Analysis for: C/C++ Software Maintenance and Evolution 50

52. Joint Master in Software Engineering Tools for OO Languages [3] Examples of tools that produce a class diagram from code  Rational Rose (Rational Software Corp.)  Paradigm Plus (Computer Associates International)  OEW (Innovative Software GmbH)  Graphical Designer (Advanced Software Technologies Inc.)  Domain Objects (Domain Objects Inc.)  COOL:Jex (Sterling Software Inc.)  Fujaba (Paderborn University)... Software Maintenance and Evolution 51

53. Joint Master in Software Engineering Example: Java Decompiler [4]  How to recover bytecode from.class file under Unix/Win with JDK? % javap -c % javap -help (to see the options)  Java Decompilers -”ClassCracker” http://www.pcug.org.au/~mayon/ -“DeCafe Pro" from DeCafe, France at http://decafe.hypermart.net/index.htm -“SourceAgain" from Ahpah corp at http://www.ahpah.com Software Maintenance and Evolution 52

54. Joint Master in Software Engineering Example: Java Decompiler [4] ClassCracker 2 Interface Software Maintenance and Evolution 53

55. Joint Master in Software Engineering Example: Java Decompiler [4]  Components of ClassCracker 2 -Java decompiler: retrieves Java source code from Java class files -Java disassembler: produces Java Assembly Code -A Java class file viewer: displays Java class file structures. Software Maintenance and Evolution 54

56. Joint Master in Software Engineering Example: Java Decompiler [4]  Features of ClassCracker 2 -User visual interface. -Can decompile class files within zip or jar files. -Conversion mode (JAVA, JASM or JDUMP) is selectable -A Batch Mode allows multiple class files to be decompiled simultaneously -more…... Software Maintenance and Evolution 55

57. Joint Master in Software Engineering Example: Java Decompiler [4]  ClassCracker 2 System Requirements - All platform (Window/Linus/Unix) - JDK /JRE Do not believe it? From myClass_origin.class ==>myClass.java % javac myClass.java (==>myClass.class) % diff myClass.class myClass_origin.class Software Maintenance and Evolution 56

58. Joint Master in Software Engineering Example: Java Decompiler [4]  ClassCracker 2.0-- want to try it? - Free download at http://www.pcug.org.au/~mayon/classcracker/ccgetde mo.html - Only first three methods are decoded.  Bridge 1.0---Free http://www.geocities.com/SiliconValley/Bridge/8617/jad.h tml Software Maintenance and Evolution 57

59. Joint Master in Software Engineering References [1] I. Sommerville, Software Engineering, Chapter 28, sixth edition, Pearson Education, 2000. [2] Reverse Engineering, Department of Mechanical Engineering, The Ohio State University, 2000. [3] J. Nummenmaa, Reverse Engineering, School of Information Sciences, the University of Tampere, www.sis.uta.fi/~jyrki/old_courses/se03/#material/reverse- engineering.ppt [4] S. Xu, Reverse Engineering, Computer Science, University of Windsor, Canada, people.auc.ca/xu/present/reverse.ppt Software Maintenance and Evolution 58

60. Tempus Thank you for your attention.