1 Compiler 3.2 A Level computing

2 So, Sir what is a compiler?

3 The Structure of a Compiler SourceTokensInterm. Language Lexical analysis Parsing Code Gen. Machine Code Today we start Optimization

4 Lexical Analysis (scanner) Scanner reads characters from the source program Scanner groups the characters into lexemes The scanner is called by parser Each lexeme corresponds to a token Example: if (i == j) z = 0; else z = 1; The input is just a sequence of characters: \tif (i == j)\n\t\tz = 0;\n\t\telse\n\t\tz = 1; lexemes

5 Sir, what the hell then you mean by Token and Lexeme ? A token is a syntactic category –In English: noun, verb, adjective, … –In a programming language: Identifier, Integer, Keyword, Whitespace, … –Given the source code I = I + rate * 60; A Java scanner would return the following sequence of tokens… IDENT ASSIGN IDENT PLUS IDENT TIMES INT-LIT SEMI-COLON tokens

6 The Parser Group of tokens into “grammatical phrases”, discovering the underlying structure of the source program Finds syntax errors. For example, in java source code I = * 5; Corresponds to the following sequence of tokens IDENT ASSIGN TIMES INT-LIT SEMI COLON All are legal tokens, but sequence of token is erroneous.

7 Once again Sir, what the hell does the parser do finally? Might find some “static semantic” errors, e.g., use of an undeclared variable, or operands that are wrongly used with variables. Might generate code, or build some intermediate representation of the program such as an abstract-syntax tree.

8 num + ()* num ‘*’ ‘(‘ num ‘+’ num ‘)’ Syntax Analyzer (Parser)

9 Semantic Analyzer Consider the Source code: I = I + rate * 60; = (float) I + (float) (float) I + (float) rate* 60 (float) rate * 60 (int)  The semantic analyzer checks for more “static semantic” errors. It may also annotate and or change the abstract syntax tree.

10 The Intermediate Code Generator The ICG translates from abstract-syntax tree to intermediate code. One possibility is 3- address code. Example: temp1 = 60 temp2 = rate * temp1 temp3 = I + temp2 I = temp3

11 The Optimizer The Optimizer tries to improve code generated by the intermediate code generator. The optimizer may also try to reduce the code smaller So.. The previous intermediate code thus becomes temp2 = rate * temp1 I = I + temp2

12 The Code Generator generates object code finally from the (optimized) intermediate code. Example: sumcalc: xorl %r8d, %r8d xorl %ecx, %ecx movl %edx, %r9d cmpl %edx, %r8d jg.L7 sall $2, %edi.L5: movl %edi, %eax cltd idivl %esi leal 1(%rcx), %edx movl %eax, %r10d imull %ecx, %r10d movl %edx, %ecx imull %edx, %ecx leal (%r10,%rcx), %eax movl %edx, %ecx addl %eax, %r8d cmpl %r9d, %edx jle.L5.L7: movl %r8d, %eax ret int sumcalc(int a, int b, int N) { int i; int x, t, u, v; x = 0; u = ((a<<2)/b); v = 0; for(i = 0; i <= N; i++) { t = i+1; x = x + v + t*t; v = v + u; } return x; }

13 So what? So the compiler converts the source program into object program. When doing this process of converting (source to object) it takes the sources code and passes it to different stages: lexical analysis, which reads the source Code character by character and coins it to form the lexemes and these lexemes then are framed into bits of tokens which are then called by Parser. The parser then groups these token into meaningful “grammatical phrases”, finds syntax errors if any and then built a intermediate representation of the program as an “abstract-syntax” tree. This now is passed through another process called Semantic analyzer, which further annotates and augments the “abstract-syntax tree”. Now this “abstract-syntax” tree is translated to an intermediate code by the Intermediate Code Generator. The outcome of the Intermediate code is further optimized by Optimizer. The optimizer can further simplify the codes and passes it to the Code Generator, which generates the desired Object code. These object code are then linked by the linkers and loaded into the memory for execution by the loader.

14 Anatomy of a Computer Step by Step (We shall see what happens) Lexical Analyzer (Scanner) Token Stream Program (character stream) Lexical Analyzer (Scanner) Token Stream Program (character stream)

15 Lexical Analyzer (Scanner) Num(234)mul_oplpar_opNum(11)add_oprpar_op 2 34 *( ) Num(-22)

16 Lexical Analyzer (Scanner) val#ue Num(234)mul_oplpar_opNum(11)add_oprpar_op 2 34 *( ) Num(-22) Not a number Variable names cannot have ‘#’ character

17 Anatomy of a Computer Lexical Analyzer (Scanner) Syntax Analyzer (Parser) Token Stream Parse Tree Program (character stream) Syntax Analyzer (Parser) Token StreamParse Tree

18 Syntax Analyzer (Parser) num + ()* num ‘*’ ‘(‘ num ‘+’ num ‘)’

19 Syntax Analyzer (Parser) int * foo(i, j, k)) int i; int j; { for(i=0; i j) { fi(i>j) return j; } Extra parentheses Missing increment Not an expressionNot a keyword

20 Anatomy of a Computer Intermediate Representation Semantic Analyzer Lexical Analyzer (Scanner) Syntax Analyzer (Parser) Token Stream Parse Tree Program (character stream) Intermediate Representation Semantic Analyzer Parse Tree

21 Semantic Analyzer int * foo(i, j, k) int i; int j; { int x; x = x + j + N; return j; } Type not declared Mismatched return type Uninitialized variable used Undeclared variable

22 Anatomy of a Computer Code Optimizer Optimized Intermediate Representation Intermediate Representation Semantic Analyzer Lexical Analyzer (Scanner) Syntax Analyzer (Parser) Token Stream Parse Tree Program (character stream) Code Optimizer Optimized Intermediate RepresentationIntermediate Representation

23 Optimizer int sumcalc(int a, int b, int N) { int i; int x, t, u, v; x = 0; u = ((4*a)b*i); v = 0; for(i = 0; i <= N; i++) { t = i+1; x = x + u + t*t; } return x; } int sumcalc(int a, int b, int N) { int i; int x, y; x = 0; y = 0; for(i = 0; i <= N; i++) { x = x+4*a/b*i+(i+1)*(i+1); } return x; }

24 Anatomy of a Computer Code Optimizer Code Generator Optimized Intermediate Representation Assembly code Intermediate Representation Semantic Analyzer Lexical Analyzer (Scanner) Syntax Analyzer (Parser) Token Stream Parse Tree Program (character stream) Code Generator Optimized Intermediate RepresentationAssembly code

25 Code Generator sumcalc: xorl %r8d, %r8d xorl %ecx, %ecx movl %edx, %r9d cmpl %edx, %r8d jg.L7 sall $2, %edi.L5: movl %edi, %eax cltd idivl %esi leal 1(%rcx), %edx movl %eax, %r10d imull %ecx, %r10d movl %edx, %ecx imull %edx, %ecx leal (%r10,%rcx), %eax movl %edx, %ecx addl %eax, %r8d cmpl %r9d, %edx jle.L5.L7: movl %r8d, %eax ret int sumcalc(int a, int b, int N) { int i; int x, t, u, v; x = 0; u = ((a<<2)/b); v = 0; for(i = 0; i <= N; i++) { t = i+1; x = x + v + t*t; v = v + u; } return x; }