2. ProgrammingLanguages Programming Languages

3. Language Syntax This lecture introduces the the lexical structure of programming languages; the context-free grammars and their description in BNF; the representation of syntactic structure using trees; the issues that arise in constructing BNFs for a programming language; the EBNFs and syntax diagrams.

4. Syntax is the structure of a language.Syntax is the structure of a language. One of the great advances in programming languages has been the development of a formal system for describing syntax that is now almost universally in use.One of the great advances in programming languages has been the development of a formal system for describing syntax that is now almost universally in use. Language Syntax In the 1950s Noam Chomsky developed the idea of context-free grammars; and John Backus, with contributions by Peter Naur, developed the Backus- Naur forms (BNFs) notational system for describing these grammars.In the 1950s Noam Chomsky developed the idea of context-free grammars; and John Backus, with contributions by Peter Naur, developed the Backus- Naur forms (BNFs) notational system for describing these grammars.

5. The lexical structure of a programming language is the structure of its words, or tokens.The lexical structure of a programming language is the structure of its words, or tokens. Typically, the scanning phase of a translator collects sequences of characters from the input program into tokens;Typically, the scanning phase of a translator collects sequences of characters from the input program into tokens; Lexical Structures which are then processed by a parsing phase, that determines the syntactic structure.which are then processed by a parsing phase, that determines the syntactic structure.

6. Tokens Typical token categories include the following:Typical token categories include the following: Reserved words, sometimes called keywords, such as "begin," "if," and "while“.Reserved words, sometimes called keywords, such as "begin," "if," and "while“. Constants or literals, such as 42 (a numeric constant) or "hello" (a string constant).Constants or literals, such as 42 (a numeric constant) or "hello" (a string constant). Special symbols, such as ";", "< =", or "+“.Special symbols, such as ";", "< =", or "+“. Identifiers, such as x24, monthly_balance, or write.Identifiers, such as x24, monthly_balance, or write.

7. Context-Free Grammars And BNFs We begin the description of grammars and BNFs with an example:We begin the description of grammars and BNFs with an example: 1. :: =.1. :: =. 2. :: = 3. ::= a | the 4. ::= girl | dog 5. :: = 6. ::= sees | pets2. :: = 3. ::= a | the 4. ::= girl | dog 5. :: = 6. ::= sees | pets In English, we can express sentences as:In English, we can express sentences as:

8. Context-Free Grammars And BNFs Thus we could construct, or derive, the sentence "the girl sees a dog." as follows:Thus we could construct, or derive, the sentence "the girl sees a dog." as follows:

9. Context-Free Grammars And BNFs A context-free grammar consist of a series grammar rules as described; the rules consist of a left-hand side that is a single structure name;A context-free grammar consist of a series grammar rules as described; the rules consist of a left-hand side that is a single structure name; followd by a right-hand side consisting of a sequence of items that can be symbols or other structure names.followd by a right-hand side consisting of a sequence of items that can be symbols or other structure names. The names for structures (like ) are called nonterminals, since they are broken down into further structures.The names for structures (like ) are called nonterminals, since they are broken down into further structures.

10. Productions The words or token symbols are also called terminals, since they are never broken down.The words or token symbols are also called terminals, since they are never broken down. Grarmmar rules are also called productions, since they "produce" the strings of the language using derivations.Grarmmar rules are also called productions, since they "produce" the strings of the language using derivations. Productions are in Backus-Naur form if they are as given using only the metasymbols ":: = ", "|", " ".Productions are in Backus-Naur form if they are as given using only the metasymbols ":: = ", "|", " ". ( Sometimes parentheses are also allowed to group things together.)( Sometimes parentheses are also allowed to group things together.)

11. Context-free? Why is such a grammar context-free?Why is such a grammar context-free? The simple reason is that the nonterminals appear singly on the left-hand sides of productions.The simple reason is that the nonterminals appear singly on the left-hand sides of productions. This means that each nonterminal can be replaced by any right-hand side alternative, no matter where the nonterminal might appear.This means that each nonterminal can be replaced by any right-hand side alternative, no matter where the nonterminal might appear. In other words, there is no context under which only certain replacements can occur.In other words, there is no context under which only certain replacements can occur.

12. Context-free? Why is such a grammar context-free?Why is such a grammar context-free? The simple reason is that the nonterminals appear singly on the left-hand sides of productions.The simple reason is that the nonterminals appear singly on the left-hand sides of productions. This means that each nonterminal can be replaced by any right-hand side alternative, no matter where the nonterminal might appear.This means that each nonterminal can be replaced by any right-hand side alternative, no matter where the nonterminal might appear. We shall adopt the view that anything not expressable using context-free grammars is a semantic, not a syntactic issue.We shall adopt the view that anything not expressable using context-free grammars is a semantic, not a syntactic issue.

13. Context-sensitivity As an example of a context-sensitivity, we noted that articles that appear at the beginning of sentences in the preceding grammar should be capitalized.As an example of a context-sensitivity, we noted that articles that appear at the beginning of sentences in the preceding grammar should be capitalized. One way of doing this is to rewrite the first rule as:One way of doing this is to rewrite the first rule as: :: = '.' :: = '.' and then add the context-sensitive rule: :: = The | Aand then add the context-sensitive rule: :: = The | A

14. Context-sensitivity (2) Now the derivation would look as follows:Now the derivation would look as follows: ->. (new rule 1) ->. (new rule 1) ->. (rule 2) ->. (rule 2) -> The. (new context-sensitive rule) -> The. (new context-sensitive rule) ->….. ->…..

15. BNF form Context-free grammars have been studied extensively by formal language theorists and are now so well understood that it is natural to express the syntax of any programming language in BNF form.Context-free grammars have been studied extensively by formal language theorists and are now so well understood that it is natural to express the syntax of any programming language in BNF form. By doing so makes it easier to write translators for the language, since the parsing stage can be automated.By doing so makes it easier to write translators for the language, since the parsing stage can be automated.

16. Syntax-directed Semantics Syntax establishes structure, not meaning.Syntax establishes structure, not meaning. But the meaning of a sentence (or program) must be related to its syntax.But the meaning of a sentence (or program) must be related to its syntax. To make use of the syntactic structure of a program to determine its semantics we must have a way of expressing this structure as determined by a derivation.To make use of the syntactic structure of a program to determine its semantics we must have a way of expressing this structure as determined by a derivation. A standard method for doing this is with a parse tree.A standard method for doing this is with a parse tree.

17. Parse Tree The parse tree describes graphically the replacement process in a derivation.The parse tree describes graphically the replacement process in a derivation. For example, the parse tree for the sentence "the girl sees a dog." is as follows:For example, the parse tree for the sentence "the girl sees a dog." is as follows:

18. A Simple Arithmetic Expression Grammar A typical simple example of the use of a context- free grammar in programming languages is the description of simple integer arithmetic expressions with addition and multiplication:A typical simple example of the use of a context- free grammar in programming languages is the description of simple integer arithmetic expressions with addition and multiplication: ::= + | * | ( ) | ::= + | * | ( ) | :: = | :: = | :: = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 :: = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

19. Ambiguity A grammar for which two distinct parse are possible for the same string is ambiguous.A grammar for which two distinct parse are possible for the same string is ambiguous. For example, if we construct: 3 + 4 * 5For example, if we construct: 3 + 4 * 5

20. Precedence and Associativity The revised disambiguating grammar for simple arithmetic expression that expresses both precedence and associativity is given as:The revised disambiguating grammar for simple arithmetic expression that expresses both precedence and associativity is given as: ::= + | ::= * | ::= ( ) | ::= | ::= 0|1|2|3|4|5|6|7|8|9 ::= + | ::= * | ::= ( ) | ::= | ::= 0|1|2|3|4|5|6|7|8|9 The above disambiguating rules define the precedence for * and + operators; and apply the left-recursive associative rule.The above disambiguating rules define the precedence for * and + operators; and apply the left-recursive associative rule.

21. EBNFs A special notation for grammar rules is adopted that expresses more clearly the repetitive nature of their structures:A special notation for grammar rules is adopted that expresses more clearly the repetitive nature of their structures: ::= { + } ::= { * } ::= ( ) | ::= { } ::= 0|1|2|3|4|5|6|7|8|9 ::= { + } ::= { * } ::= ( ) | ::= { } ::= 0|1|2|3|4|5|6|7|8|9 We assume that any operator involved in a curly bracket repetition is left-associative.We assume that any operator involved in a curly bracket repetition is left-associative.

22. Syntax Diagrams A useful graphical representa tion for a grammar rule is the syntax diagram.A useful graphical representa tion for a grammar rule is the syntax diagram.