XQuery from the Experts Chapter 5 – Introduction to the formal Semantics Νίκος Λούτας

Outline Getting started with the formal semantics Dynamic Semantics Environments Matching Values and Types Errors Static Semantics Type Soundness Evaluation order Normalization

Outline (cont’d) Learning more about XQuery Values and Types Matching and Subtyping FLWOR Expressions Path Expressions Implicit Coercion and Function calls Node Identity and Element Constructors

Getting Started… The XQuery formal semantics describes a processing model that relates: Query parsing Takes as input a query and produces a parse tree Normalization Transforms the parse tree in an equivalent parse tree in the core language Static analysis Produces a parse tree where each expression has been assigned a type Dynamic evaluation Take a parse tree in the core language and reduces its expression to XML values  that is the result of the query

Dynamic Semantics Evaluation  takes an expression and returns a value Expr  Value Value ::= Boolean | Integer Boolean ::= fn: true() | fn: false() Integer ::= 0 | 1 | -1 | 2 | -2 |…

Dynamic Semantics (cont’d) Expr ::= Value |Expr < Expr |Expr + Expr |if (Expr) then Expr else Expr e.g. 5 < 10, 1 + 2, if(1 < 2) then else 6 + 7

Dynamic Semantics (cont’d) Evaluation is described by five rules: i. Value  Value(VALUE) ii. Expr 0  Integer 0 Expr 1  Integer 1 Expr 0 < Expr 1  Integer 0 < Integer 1 (LT) iii. Expr 0  Integer 0 Expr 1  Integer 1 Expr 0 + Expr 1  Integer 0 + Integer 1 (SUM)

Dynamic Semantics (cont’d) iv. Expr 0  fn: true() Expr 1  Value if (Expr 0 ) then Expr 1 else Expr 2  Value (IF-TRUE) iv. Expr 0  fn: true() Expr 1  Value if (Expr 0 ) then Expr 1 else Expr 2  Value (IF-FALSE)

Dynamic Semantics (cont’d) Example: Build a proof tree to evaluate the expression  1 (VALUE) 2  2 (VALUE)  3 (SUM)

Environment dynEnv├ Expr  Value dynEnv  dynamic Environment An environment may have many components varValue  a map from variables to their values Binding a variable to an environment overrides any previous bindings of the variable

Environment (cont’d) NotationMeaning  The initial environment with an empty map dynEnv.varValue (Var 1  Value 1,…, Var n  Value n ) The environment that maps Var i to Value i dynEnv + varValue (Var  Value) The environment identical to dynEnv except that maps Var to Value dynEnv.varValue (Var)The value of var in dynEnv dom(dynEnv.varValue) The set of variables mapped in dynEnv

Environment (cont’d) Expr ::= …previous expressions… |$Var |let $Var := Expr return Expr e.g. $x, let $x := 1 return $x + 2

Environment (cont’d) The five rules shown before need to be revised, e.g. (LT) ii. dynEnv├ Expr 0  Integer 0 dynEnv├ Expr 1  Integer 1 dynEnv├ Expr 0 < Expr 1  Integer 0 < Integer 1 two more rules are added vi. dynEnv.varValue (Var) = Value dynEnv├ $Var = Value(VAR) vii. dynEnv├ Expr 0  Value 0 dynEnv + varValue (Var  Value 0 ) ├ Expr 1  Value 1 dynEnv├ let $Var := Expr 0 return Expr 1  Value 1 (LET)

Matching Values and Types The value must match the variables type, else an exception is raised Expr ::= …previous expressions… |let $Var as Type := Expr return Expr Type ::= xs: boolean | xs: integer Static type declarations  when the expression is analyzed Dynamic type declarations  when the expression is evaluated Value matches Type

Matching Values and Types (cont’d) Three new rules derive viii. Integer matches xs: Integer(INT-MATCH) ix. Boolean matches xs: Boolean (BOOL-MATCH) vi. dynEnv├ Expr 0  Value 0 Value 0 matches Type dynEnv + varValue (Var  Value 0 ) ├ Expr 1  Value 1 dynEnv├ let $Var as Type := Expr 0 return Expr 1  Value 1 (LET-DECL)

Errors dynEnv├ Expr raises Error Error ::= typeErr | dynErr Expr ::= …previous expressions… |Expr idiv Expr

Errors (cont’d) Type errors  triggered if an operand’s value does not match the operator’s required type not (Value matches Type) dynEnv├ Expr 0  Value 0 not (Value 0 matches Type) dynEnv├ let $Var as Type := Expr0 return Expr1 raises typeErr

Errors (cont’d) - Dynamic errors dynEnv├ Expr 0  Value 0 dynEnv├ Expr 1  Value 1 Value 1 ≠ 0 dynEnv├ Expr 0 idiv Expr 1  Value 0 idiv Value 1 dynEnv├ Expr 1  0 dynEnv├ Expr 0 idiv Expr 1 raises dynErr

Errors (cont’d) Example: what errors does the following expression raise? (1 idiv 0) + (2 < 3) typeErrdynErr

Static Semantics How static types associated with expressions Static typing  takes a static environment and an expression and returns a type statEnv ├ Expr : Type statEnv  the static environment that captures the context available at query-analysis time (variables and their types) No need to check for type errors at evaluation time

Static Semantics (cont’d) Two rules to assign type statEnv├ Boolean : xs: boolean (BOOLEAN-STATIC) statEnv├ Integer : xs: integer (INTEGER-STATIC) statEnv├ Expr 0 : xs: boolean statEnv├ Expr 1 : Type statEnv├ Expr 2 : Type statEnv├ if (Expr 0 ) then Expr 1 else Expr 2 : Type (IF-STATIC) We do not examine the value of the Expr – the value is not known statically Examine only the type of the condition  must be boolean The branches must have the same type

Static Semantics (cont’d) Expression: if (1 < 3) then else statEnv ⊢ 1 : integer statEnv ⊢ 3 : integer (BOOLEAN-STATIC) statEnv ⊢ 1 < 3 : boolean (INTEGER-STATIC) statEnv ⊢ : integer (IF-STATIC) statEnv ⊢ if (1 < 3) then else : integer statEnv ⊢ 3 : integer statEnv ⊢ 4 : integer (INTEGER-STATIC) statEnv ⊢ : integer statEnv ⊢ 5 : integer statEnv ⊢ 6 : integer

Type Soundness Suppose Expr : Type  Expr either yields a value of the same type or raises a dynamic error dynEnv matches statEnv  capture the relationship between dynEnv and statEnv dynEnv 1 := varValue (x  1, y  0, z  fn: false()) statEnv 1 := varType (x: xs: integer, y: xs: integer, z: xs: boolean)

Type Soundness (cont’d) Theorem for Values if dynEnv matches statEnv dynEnv ├ Expr  Value statEnv ├ Expr  Type then Value matches type

Type Soundness (cont’d) Example dynEnv 1 matches statEnv 1 dynEnv 1 ├ if ($z) then $x else $y  0 statEnv 1 ├ if ($z) then $x else $y : xs: integer 0 matches xs: integer

Type Soundness (cont’d) Theorem for Errors if dynEnv matches statEnv dynEnv ├ Expr raises Error statEnv ├ Expr : Type then Error ≠ typeErr

Type Soundness (cont’d) Example dynEnv 1 matches statEnv 1 dynEnv 1 ├ $x idiv $y raises dynErr statEnv 1 ├ $x idiv $y : xs: integer

Type Soundness (cont’d) Remember that If an expression raises a type error, then it cannot type check e.g. dynEnv 1 ├ $x + $z raises typeErr statEnv 1 ├ $x + $y : Type An expression that does not raise a type error may still fail to statically type dynEnv 1 ├ if ($x < $y) then $x + $z else $y  0 statEnv 1 ├ if ($x < $y) then $x + $z else $y : Type

Evaluation Order Test the expressions in either order Expr and Expr Stop and return false if either one is false Raise an error if either one raises an error Example (1 idiv 0 < 2) and (4 < 3) Two possible results: dynErr or false Depends on which will be evaluated first Both correct 11

Normalization takes an expression in full XQuery and returns an equivalent expression in core XQuery [FullExpr] Expr == Expr FullExpr ::= Expr | let $Var as Type := Expr where Expr return Expr [Expr 0 + Expr 1 ] Expr == [Expr 0 ] Expr + [Expr 0 ] Expr [$Var ] Expr = $Var [let $Var as Type := Expr 0 where Expr 1 return Expr 2 ] Expr == let $Var as Type := [Expr 0 ] Expr return if ([Expr 1 ] Expr ) then [Expr 2 ] Expr else ()

Outline So far we have covered: Dynamic Semantics Environments Matching Values and Types Errors Static Semantics Type Soundness Evaluation order Normalization

Outline (cont’d) Now we will talk in more depth about: Values and Types Matching and Subtyping FLWOR Expressions Path Expressions Implicit Coercion and Function calls Node Identity and Element Constructors

Part II: Values and Types Value  sequence of one or more items Value ::= () | Item (,Item)* Item ::= AtomicValue | NodeValue AtomicValue ::= xs: integer(String) | xs: boolean(String) | xs: string(String) | xs: date(String) e.g. xs: string(“XQuery’)(  XQuery), xs:boolean(“false”) (  fn: false())

Values and Types (cont’d) NodeValue ::= element ElementName TypeAnnotation? { Value } |text { String } ElementName ::= QName TypeAnnotation ::= of type TypeName TypeName ::= QName

Values and Types (cont’d) ItemType ::= NodeType | AtomicType NodeType ::= ElementType |text () AtomicType ::= AtomicTypeName AtomicTypeName ::= xs:string | xs:integer | xs:boolean | xs:date

Values and Types (cont’d) ElementType := element((ElementName (,TypeName)?)?) element(article)  global declaration element(article, xs:string)  local declaration Type ::= none() | empty() | ItemType | | Type, Type | Type | Type | |Type Occurrence Occurrence ::= ? | + | *

Values and Types (cont’d) SimpleType ::= AtomicTypeName | SimpleType | SimpleType | SimpleType Occurrence Definition ::= define element ElementName TypeAnnotation | define type TypeName TypeDeriviation TypeDeriviation ::= restricts AtomicTypeName | restricts TypeName { Type } | { Type }

Values and Types (cont’d) Example define element article of type Article define type Article { element (name, xs; string), element (reserve_price, PriceList) * } define type PriceList restricts xs:anyType { xs:decimal *}

Matching and Subtyping Matching  relate complex XML values with complex types e.g element reserve_price of type PriceList {10.0, 20.0, 25.0} matches element (reserve_price) Subtyping  checks whether a type is a subtype of another

Matching and Subtyping (cont’d) Yields ElementType yields element (ElementName,TypeName) ElementType Reference to a global element  name of the element and type annotation from the element declaration Contains an element name with a type annotation  element name and type name in the type annotation Has a wildcard name followed by type name  wildcard name and type name Has neither element name nor type name  wildcard name and xs:anyType

Matching and Subtyping (cont’d) Substitutes for ElementName 1 substitutes for ElementName 2 When the two names are equal When the second name is the * An element name may substitute for itself statEnv├ ElementName substitutes for ElementName

Matching and Subtyping (cont’d) Derives TypeName 1 derives from TypeName 2 e.g. PriceList derives from xs:anyType Every type name derives derives from the type name that is declared to derive from by restriction Reflexive and transitive

Matching and Subtyping (cont’d) Matches Value matches Type e.g. (10.0, 20.0, 25.0) matches xs:decimal * The empty sequence matches the empty sequence, e.g. () matches (). If two values match two types, then their sequence matches the corresponding sequence type.

Matching and Subtyping (cont’d) Matches If a value matches a type, then it also matches a choice type, where that type is one of the choices. Value matches Type 1 Value matches Type 1 | Type 2 A value matches an optional occurrence of a type of it either matches the type or the empty sequence Value matches empty() | Type Value matches Type ?

Matching and Subtyping (cont’d) Subtyping Type 1 subtype Type 2 If and only if Value matches Type 1  Value matches Type 2 e.g. element(*, PriceList) subtype element(xs:integer)

FLWOR Expressions Expr ::= …previous expressions… | FLWRExpr FLWRExpr ::= Clause+ return Expr Clause ::= ForExpr | LetExpr | WhereExpr ForExpr ::= for ForBinding (, ForBinding) * WhereExpr ::= where Expr LetExpr ::= let LetBinding (, LetBinding) * ForBinding ::= $Var TypeDeclaration? PositionVar? in Expr LetBinding ::= $Var TypeDeclaration? := Expr TypeDeclaration ::= as SequenceType PositionVar ::= at $Var SequenceType ::= ItemType Occurrence

FLWOR Expressions (cont’d) Normalization A for / let clause with more than one binding turns each binding into a separate nested for / let expression and normalizes the result, (n>1) [let LetBinding 1 … LetBinding n return Expr] Expr == [let LetBinding 1 return … [ for LetBinding n return Expr] Expr ] Expr a where clause is normalized into an if expression that returns the empty sequence if the condition is false, and normalizes the result [where Expr 0 return Expr 1 ] Expr == [if(Expr 0 ) then Expr 1 else ()] Expr

FLWOR Expressions (cont’d) for $i in $I, $j in $J let $k := $i + $j where $k >= 5 return ($i, $j) for $i in $I return for $j in $J return let $k := $i + $j if ($k >= 5) then ($i,$j) else () Normalization

FLWOR Expressions (cont’d) Factored types  consist of an item type and an occurrence indicator Result type = Prime ∙ Quantifier e.g. ((xs:integer, xs:string) | xs:integer) * subtype (xs:integer | xs:string) * prime ((xs:integer, xs:string) | xs:integer) * = xs:integer | xs:string quant ((xs:integer, xs:string) | xs:integer) * = *

FLWOR Expressions (cont’d) Factorization theorem for all types we have Type subtype prime (Type) ∙ quant (Type) further if Type subtype Prime ∙ Quantifier then prime (Type) subtype Prime and quant (Type) ≤ Quantifier 1 ≤ ?, 1 ≤ +, ? ≤ *, + ≤ *

Path Expressions [QName] Path == child:: QName [book/isbn] Path == child:: book/child:: isbn [.] Path == self:: node() [..] Path == parent:: node() [Expr 1 //Expr 2 ] Path == [Expr 1 /descendant-or-self:: node()/Expr 2 ] Path

Path Expressions (cont’d) Expr ::= …previous expressions… | PathExpr PathExpr ::= / | / RelativePathExpr | RelativePathExpr RelativePathExpr ::= RelativePathExpr / StepExpr | StepExpr | RelativePathExpr // StepExpr StepExpr ::= (ForwardStep | ReverseStep) Predicates ForwardStep ::= ForwardAxis NodeTest ReverseStep ::= ReverseAxis NodeTest ForwardAxis ::= child:: | descendant:: | self:: | descendant-or-self:: ReverseAxis ::= parent:: Predicates ::= ( [ Expr ] )* NodeTest ::= text() | node() | * | QName

Rule that relates normalization of expressions to normalization of path expressions: [PathExpr] Expr == fs:distinct-docorder ([PathExpr] Path ) Normalization of absolute path expressions [/] path == fn:root($fs:dot) [/RelativePathExpr] path == [ fn:root($fs:dot) /RelativePathExpr] path Built-in variable $fs:dot represents the context node An absolute path expression refers to the root of the XML tree that contains the context node Path Expressions (cont’d)

Normalization of “/” [RelativePathExpr / StepExpr] path == let $fs:sequence := fs:distinct- docorder( [RelativePathExpr] path ) return let $fs:last := fn:count($fs:sequence) return for $fs:dot at $fs:position in $fs:sequence return [StepExpr] path This rule binds the variables $fs:sequence, $fs:last, $fs:dot and $fs:position to, respectively, the context sequence, the context size, the context node and the position of that node in the context sequence

Path Expressions (cont’d) Normalization of step expressions: [ForwardStep Predicates [Expr]] Path == let $fs:sequence := [ForwardStep Predicates] Path return let $fs:last := fn:count($fs:sequence) return for $fs:dot at $fs:position in $fs:sequence return if ( [Expr] Predicates ) then $fs:dot else () Similar rule for ReverseStep but the $fs:position is bound reversely Example (simplified): child::*[2]  let $fs:sequence := child::* return let $fs:last := fn:count($fs:sequence) return for $fs:dot at $fs:position in $fs:sequence return if (fn:position() = 2) then $fs:dot else ()

Path Expressions (cont’d) Predicate mapping [Expr] Predicates == typeswitch( [Expr] Expr ) case numeric $v return op:numeric-equal(fn:round($v), $fs:position) default $v return fn:boolean($v) Finally, axis mapping is straightforward [ForwardAxis :: NodeTest] Path == ForwardAxis :: Nodetest [ReverseAxis :: NodeTest] Path == ReverseAxis :: Nodetest

Path Expressions (cont’d) path expression $input//a/b is normalized to fs:distinct-docorder( let $fs:sequence := ( fs:distinct-docorder( let $fs:sequence := $input return let $fs:last := fn:count($fs:sequence) return for $fs:dot at $fs:position in $fs:sequence return fs:distinct-docorder( let $fs:sequence := descendant-or-self::node() return let $fs:last := fn:count($fs:sequence) return for $fs:dot at $fs:position in $fs:sequence return child::a)) ) return let $fs:last := fn:count($fs:sequence) return for $fs:dot at $fs:position in $fs:sequence return child::b )

Implicit Coercion and Function calls XQuery can represent a Schema containing irregular data in the formal type notation define element article of type Article define type Article { element (name, xs: string), element (reserve_price, PriceList) *, } define type PriceList { xs: decimal * }

Implicit Coercion and Function calls (cont’d) An arithmetic expression is well defined on any item sequence that can be coerced to zero or one atomic value $article/reserve_price  a sequence of zero or more reserve_price elements A comparison is well defined on any item sequence that can be coerced to a sequence of atomic values $article/reserve_price < 100  the typed context of $article/reserve_price is automatically extracted XPath’s predicate expressions are well defined on any item sequence $article[reserve_price]  returns each node in $article that has at least one reserve_price child

Implicit Coercion and Function calls (cont’d) First coercion Applied to expressions that require a boolean value Maps the Expr argument to a core expression and applies fn: boolean to the result [if (Expr 0 ) then Expr 1 else Expr 2 ] Expr == if (fn: boolean([Expr 0 ]Expr)) then [Expr 1 ] Expr else [Expr 2 ] Expr

Implicit Coercion and Function calls (cont’d) Second coercion Applied to an expression when used in a context that requires a sequence of atomic values Maps the Expr argument to a core expression then applies the fn: data to the result fn: data  takes any item sequence, applies the following rules and concatenates the results If the item is an atomic value, it is returned Otherwise, the item is a node and its typed value is returned

Implicit Coercion and Function calls (cont’d) Normalization rule for + [Expr 1 + Expr 2 ] Expr == let $v1 := fn: data([Expr 1 ] Expr ) return let $v2 := fn: data([Expr 2 ] Expr ) return fs: plus($v1, $v2) Normalization rule for < [Expr 1 < Expr 2 ] Expr == some $v1 in fn: data([Expr 1 ] Expr ) satisfies some $v2 in fn: data([Expr 2 ] Expr ) satisfies fs: less-than($v1, $v2) $article/reserve_price ( )  returns () ( ( )  returns ( ( ), ( )  type error, because the atomized value is a sequence of decimals

Node Identity and Element Constructors Element Constructor  creates new nodes with new identities e.g let $name := Red Bicycle return {$name, $name} Store  mapping from node identifiers to node values Item ::= NodeId | AtomicValue dynEnv  store(NodeId  NodeValue)

Node Identity and Element Constructors (cont’d) Red Bicycle Store( N1  element article of type Article {N2, N3, N4, N5}, N2  element article of type xs:string {“Red Bicycle”}, N3  element article of type xs:date {“ ”}, N4  element article of type xs:date {“ ”}, N5  element article of type xs:decimal {“40”})

Node Identity and Element Constructors (cont’d) Evaluation affects the store Store 0 ; dynEnv├ Expr  Value ; Store 1 Each new store computed through a given judgment is passed as input to the next judgment Most rules treat the store implicitly dynEnv├ Expr 0  Value 0 dynEnv + varValue(Var  Value 0 ) ├ Expr 1  Value 1 dynEnv├ let $Var := Expr 0 return Expr 1  Value 1 ; Store 2 ; Store 1 Store 1 ; Store0 ;

Node Identity and Element Constructors (cont’d) Validation Evaluate the expression to yield a value Erase all type information in the value Construct the untyped element node Validate the node to yield the final typed value Schema Untyped document all nodes have an associated type annotation

Node Identity and Element Constructors (cont’d) Static semantics and element construction The static type system performs a conservative analysis that catches errors early, during static analysis rather than dynamic evaluation. e.g. if the element is declared to have type xs: integer then the type of its contents must be xs: integer