1. Positional Grouping in XQuery Michael Kay

2. 2 Definitions Grouping –a function that takes a sequence as input, and produces a sequence of sequences as output Value-based Grouping –assignment of items to groups is based on properties of the item Positional Grouping –Identification of structure based on patterns in the sequence of items

3. 3 Requirements Based on use cases Real-life problems gathered from xsl-list 1999-2005 Some were captured in the XSLT 2.0 requirements (2001), others came later No attempt to define any theoretical notion of completeness, e.g. a class of grammars

4. 4 Use Case 1: Headings and Paragraphs Morning Got up Made tea Afternoon Had lunch Fed the cat Posted a letter Got up Made tea Had lunch Fed the cat Posted a letter

5. 5 Use Case 2: Adjacent Bullets one two one two

6. 6 Use Case 3: Term Definition Lists XML Extensible Markup Language XSLT XSL Transformations A language for transforming XML A specification produced by W3C XML Extensible Markup Language XSLT XSL Transformations A language for transforming XML A specification produced by W3C

7. 7 Use Case 4: Continuation Markers One way to understand positional grouping is as an exercise in parsing. To get from a sequence of items to a tree, we could use some kind of grammar. One way to understand positional grouping is as an exercise in parsing. To get from a sequence of items to a tree, we could use some kind of grammar.

8. 8 Use Case 5: Page ranges 4, 6, 9, 11, 12, 13, 18, 20, 21 4, 6, 9, 11-13, 18, 20-21

9. 9 Use Case 6: Arrange in rows "Green", "Pink", "Lilac", "Turquoise", "Peach", "Opal", "Champagne" Green Pink Lilac Turquoise Peach Opal Champagne

10. 10 Use Case 7: Level Numbers Michael Kay mike@saxonica.com Norm Walsh norm@nwalsh.com Michael Kay mike@saxonica.com Norm Walsh norm@nwalsh.com

11. 11 XQuery 1.0 Solutions Head/tail recursion Positional indexing

12. 12 "Headings and Paragraphs" using head/tail recursion declare function local:section($e as element(H2)) { { local:nextPara($e/following-sibling::*[1][self::P]) } }; declare function local:nextPara($p as element(P)?) { if ($p) then ($p, local:nextPara($p/following-sibling::*[1][self::P])) else () }; {for $h in doc('doc.xml')//BODY/H2 return local:section($h)}

13. 13 "Term Definition Lists" using positional indexing let $s := (for $e at $p in $input where $e[self::dt and not(preceding-sibling::*[1][self::dt])] return $p, count($input)+1) for $i in 1 to count($s) - 1 return { for $j in $s[$i] to $s[$i + 1] - 1 return $input[$j] }

14. 14 XSLT 2.0 approach handles both value-based and positional grouping But the two are largely distinct Three varieties of positional grouping: –group-starting-with –group-ending-with –group-adjacent

15. 15 Identifying Breaks All use cases have these properties: –input sequence is the concatenation of the output sequence –"breaks" depend only on: the item before the break the item after the break position (use case "arrange in rows" only)

16. 16 Conceptual approach to solution Higher order function partition( $population as item()*, $break-function as function($after as item(), $before as item(), $position as xs:integer) as xs:boolean(), $action as function($group as item()*) as item()* ) as item()*

17. 17 Syntactic realisation partition $g in population break after $a before $b at $p where condition return action

18. 18 Solution to Use Case 1: Headings and Paragraphs Morning Got up Made tea Afternoon Had lunch Fed the cat Posted a letter Got up Made tea Had lunch Fed the cat Posted a letter partition $section in body/* break before $e where $e[self::h2] return { for $p in $section/p return {$p} }

19. 19 Solution to Use Case 2: Adjacent Bullets one two one two partition $children in * break after $a before $b where not($a[self::bullet] and $b[self::bullet]) return if ($children/self::bullet) then {$children} else $children

20. 20 Use Case 3: Term Definition Lists XML Extensible Markup Language XSLT XSL Transformations A language for transforming XML A specification produced by W3C XML Extensible Markup Language XSLT XSL Transformations A language for transforming XML A specification produced by W3C partition $term in * break after $a before $b where ($a[self::dd] and $b[self::dt]) return {$term}

21. 21 Use Case 4: Continuation Markers One way to understand positional grouping is as an exercise in parsing. To get from a sequence of items to a tree, we could use some kind of grammar. One way to understand positional grouping is as an exercise in parsing. To get from a sequence of items to a tree, we could use some kind of grammar. partition $para in./in break after $a where not($a/@cont = "yes") return {$para}

22. 22 Use Case 5: Page ranges 4, 6, 9, 11, 12, 13, 18, 20, 21 4, 6, 9, 11-13, 18, 20-21 partition $range in $page-numbers break after $a before $b where ($b != $a + 1) return if (count($range) = 1) then $range else concat($range[1], "-", $range[last()]

23. 23 Use Case 6: Arrange in rows "Green", "Pink", "Lilac", "Turquoise", "Peach", "Opal", "Champagne"......... partition $rows in $colours break at $p where (($p - 1) mod 3 = 0) return { for $i in $rows return {$i} }

24. 24 Use Case 7: Level Numbers declare function f:group( $items as element()*, $level as xs:integer) as element() { partition $group in $items break before $b where $b/@level = $level return element {$group[1]/node-name()} { f:group(remove($group, 1), $level + 1) } }; f:group(/data/*, 0)

25. 25 Performance Algorithm is intrinsically O(n) Memory usage: –naive implementation uses memory proportional to size of largest group –smart implementation can be fully streamed Almost inevitably better than the XQuery 1.0 solutions

26. 26 Conclusions Need exists for both value-based and positional grouping Positional grouping use-cases can be solved by identifying breaks in terms of (before, after, position) Conceptual approach based on higher-order functions, realised in concrete syntax