1. Introduction to Language Processing Technology Natawut Nupairoj, Ph.D. Department of Computer Engineering Chulalongkorn University

2. Outline Level of Programming Languages. Language Processors. Specification of Programming Languages.

3. swap(int v[], int k) { int temp; temp = v[k]; v[k] = v[k+1]; v[k+1] = temp; } swap: muli $2, $5, 4 add $2, $4, $2 lw $15, 0($2)... Assembler C Compiler Level of Programming Languages 000010001101101100110000... High level: C / Java / Pascal Low level: Assembly / Bytecode Machine Language

4. High-Level Language Characteristics Expressions: a = b + (c – d)/2; Data types:  Integer, character, boolean.  Record, array. Control structures:  Selective.  Iterative.

5. High-Level Language Characteristics Declarations:  Identifier can be constant, variable, procedure, function, and type. Abstraction:  Object-oriented concept.  Concern only what, not how. Encapsulation:  Object-oriented concept.  Information hiding.

6. Language Processors Program that manipulates programs express in some programming languages. Example:  Editor.  Translator / Compiler.  Interpreter.

7. Translator Translate a “source” program into an “equivalent” “object” program. Translator source program object program error messages C C++ FORTRAN Java VB Assembly C Bytecode p-code

8. Tombstone Diagrams Ordinary program Program P Written with Language L L P Java Sort x86 Sort x86 Web Browser

9. x86 Web Browser Tombstone Diagrams Machine M Machine M x86 SPARCx86 SPARC x86 Web Browser

10. Tombstone Diagrams Translator L S  T S is translated to T Translator is written with Language L C Java  x86 x86 Java  x86 C++ Java  C

11. Tombstone Diagrams Compilation x86 C  x86 x86 Sort C x86 Sort

12. Tombstone Diagrams Cross Compilation x86 C  SPARC x86 SPARC Sort SPARC Sort C

13. Tombstone Diagrams x86 Java  C x86 C  x86 x86 Two-stage compilation C Sort Java Sort x86 Sort

14. Tombstone Diagrams x86 C  x86 x86 Compiling a compiler C Pascal  x86 x86 Pascal  x86

15. Tombstone Diagrams Interpreter SLSL Interpret source S x86 Written in language L Basic x86 Basic x86 SQL SPARC Basic Sort

16. Tombstone Diagrams Abstract machine = hardware emulator  interpreter for low-level language. x86 C  x86 x86 370 C 370 x86 370 x86 = 370 HW1 370 HW1

17. Tombstone Diagrams Java  Portable environment: write-once-run-anywhere.  Interpretive compiler. M Java  JVM JVM M JVM = Bytecode

18. Tombstone Diagrams x86 JVM x86 SPARC JVM SPARC JVM Sort JVM Sort x86 Java  JVM x86 JVM Sort Java Sort

19. Tombstone Diagrams Bootstrapping  Compiler L that is written on L language. Full bootstrap  Start from nothing. Half bootstrap  Start from other machine. NNP C  NNP

20. Tombstone Diagrams Full Bootstrap NNP C sub C sub  NNP NNP C sub  NNP NNP C sub C  NNP NNP C  NNP NNP C sub  NNP NNP C sub  NNP NNP C sub  NNP

21. Tombstone Diagrams NNP C C  NNP NNP C  NNP NNP C  NNP

22. Tombstone Diagrams NNP C sub C sub  NNP NNP C sub  NNP NNP C sub C  NNP NNP C  NNP NNP C sub  NNP NNP C  NNP NNP C C  NNP

23. Tombstone Diagrams Half Bootstrap x86 C  x86 x86 C C  NNP x86 C  NNP x86 C  NNP x86 C C  NNP NNP C  NNP x86 C  X86 x86

24. Specification of Programming Language Specification  Syntax Define symbol and structure of the language. Grammar.  Contextual constraints Constraints beyond grammar. Rules of the language: scope rules, type rules, etc.  Semantics Meaning of program: its behaviors when run. How to translate a sentence S of the language L to a machine code on M

25. Syntax Context-free grammar  Terminals.  Non-terminals / Variables.  Start symbol.  Production rules. Usually being expressed with BNF notation.

26. BNF Notation Backus-Naur Form. Given production rule: N   N   Can be written as: N ::= 

27. Example: Mini-Triangle Program ! This is a comment. It continues to the end-of-line. let const m ~ 7; var n: Integer in begin n:= 2 * m * m; putint(n); end Terminals beginconstdoelseendif inletthenvarwhile ;::=~() +-*/<> =\

28. Mini-Triangle Syntax Program::=Command Command::=single-Command |Command ; single-Command single-Command ::=V-name := Expression |Identifier ( Expression ) |if Expression then single-Command else single-Command |while Expression do single-Command |let Declaration in single-Command |begin Command end

29. Mini-Triangle Syntax Expression::=primary-Expression |Expression Operator primary-Expression primary-Expression ::=Integer-Literal |V-name |Operator primary-Expression |( Expression ) V-name::=Identifier Declaration::=single-Declaration |Declaration ; single-Declaration single-Declaration ::=const Identifier ~ Expression |var Identifier : Type-denoter

30. Mini-Triangle Syntax Type-denoter::=Identifier Operator::=+ | - | * | / | | = | \ Identifier::=Letter | Identifier Letter |Identifier Digit Integer-Literal ::=Digit | Integer-Literal Digit Comment::=! Graphic* eol Letter::=a | b | … |z Digit::=0 | 1 | 2 | … | 9

31. Syntax Tree Ordered tree with  Internal nodes: non-terminals.  Leaf nodes: terminals.  N-tree of G is a syntax tree with N as the root.

32. Mini-Triangle Syntax Tree Expression ::= primary-Expression | Expression Operator primary-Expression primary-Expression ::= Integer-Literal | V-name | Operator primary-Expression |( Expression ) V-name ::= Identifier …