XSLT and XQuery - fgeorges.orgfgeorges.org/papers/fgeorges-xmlss-xslt-trends-2016.pdfIntroduction • XPath 2.0, XSLT 2.0 and XQuery 1.0 have been released in 2007 • XPath and XSLT

www.xmlsummerschool.com

summer school

Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License

Trends in XSLT and XQuery

Florent Georges

H2O Consulting

XSLT and XQuery16 September 2016

summer school

www.xmlsummerschool.com Slide 2Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License

Learning Objectives

1. Understand the versions of the standards2. Learn new features3. Practice them

Get an overview of new features and possibilities ofthe new version of XPath, XSLT and XQueryBoth by learning theory and by trying them outdirectly in exercises

http://www.xmlsummerschool.com/

summer school


Contents

Lecture1. Introduction2. XPath 3.13. XQuery 3.14. XSLT 3.0

Tutorial Session



summer school


Introduction

Fashion overview

summer school


Introduction

• XPath is a language on its own• With a standard library [F&O]• XQuery is a super set of XPath• XSLT uses XPath as its expression language• They are all functional languages (some prefer

to say functional-oriented)• They all use the same data model: XDM• They all use XML Schema as their data type

definition language


summer school


Introduction

• XPath 2.0, XSLT 2.0 and XQuery 1.0 have beenreleased in 2007

• XPath and XSLT 2.0 were a big improvementover the version 1.0

• XQuery was a “version-1.0 language”, but withall the past experience of XPath and XSLT

• Since then, we gathered more experience insome fields, like XML databases, and inapplications written entirely in those languages

• Time for new version with major improvements


summer school


Introduction

All languages will now be aligned to the sameversion number:

3.0


summer school


Introduction

Well, some are 3.1 and some are 3.0...



summer school


XPath 3.1

The basics and the commons

summer school


XPath 3.1

1. New convenience operators2. New functions3. EQNames4. Variable declaration5. Union types6. Function items7. Inline functions8. Higher-order functions9. Partial application


summer school


XPath – Convenience operators

• Concat operator: '…' || '…'

'Hello, ' || $you || '!'

==> 'Hello, world!'

concat('Hello, ', $you, '!')

==> 'Hello, world!'


summer school



• Mapping operator: lhs ! rhs

(: path operator is not applicable to atomic values :) (1, 2) / string(.) (: path operator is not (always) equivalent to a for loop :)

(: what does the following do? :) for $n in $nodes return string($n)

(: what does the following do? :) $nodes / string(.)


summer school



• The mapping operator “lhs ! rhs” is equivalentto “for $i in lhs return rhs”, with “$i” set as thecontext item of “rhs”

• Need to review XDM, anyone?

(: what does the following do? :) for $i in $items return string($i)

(: what does the following do? :) $items ! string(.)


summer school


XPath – New functions

• round()• format-date(), format-time()• format-number()• generate-id()• parse-xml(), serialize()• analyse-string()• Math: exp(), pi(), sqrt(), cos(), log10(), etc.• New utility higher-order functions, usual

functional utilities:– fold-left(), fold-right(), filter(), etc.


summer school


XPath – EQNames

• There is no way to bind namespace URIs toprefixes in XPath itself

• EQNames provide the namespace URI directlyin the expression

• Expressions are self-contained

(: what does the following mean? :) /db:article/db:info/db:title

(: what does the following mean? :) /Q{http://docbook.org/ns/docbook}article /Q{http://docbook.org/ns/docbook}info /Q{http://docbook.org/ns/docbook}title


http://docbook.org/ns/docbook

http://docbook.org/ns/docbook

summer school


XPath – Variable declaration

• XPath expressions are sometimes convoluted,if not impossible at all, because it is notpossible to define variables

• XPath 3.0 includes let expression from XQuery

(: use variables, for readability :) let $amount := 1000 let $rate := 0.21 let $vat := $amount * $rate let $included := $amount + $vat return 'Amount is ' || $amount || ', with VAT: ' || $included


summer school


XPath – Union types

• Function parameters can be typed• Have to pick the most specialized super-type• Sometimes you want to say “type A or type B”• That is possible with union types

(: use variables, for readability :) let $amount := 1000 let $rate := 0.21 let $vat := $amount * $rate let $included := $amount + $vat return 'Amount is ' || $amount || ', with VAT: ' || $included


summer school


XPath – Union types

<xsl:import-schema> <xs:schema targetNamespace="http://fgeorges.org/tmp/xmlss"> <xs:simpleType name="int-or-str"> <xs:union memberTypes="xs:integer xs:string"/> </xs:simpleType> </xs:schema> </xsl:import-schema>

<xsl:template match="/"> <is-union> <xsl:value-of select="1 instance of fg:int-or-str"/> </is-union> </xsl:template>


summer school


XPath – Function items

• Functions are first-class citizens in XPath!• We can obtain an item which represents a

function, pass it around, call it dynamically• my:function#2 returns a function items for the

function my:function() with arity 2 (that is,with 2 parameters)

• $fun('world') calls the function referenced bythe function item in $fun (must be arity 1)

• New ItemType for function items:function(xs:string?, element()+)


summer school


XPath – Function items

(: function item for my:less-than($lhs, $rhs) :) (: function items do not have a string value :) my:less-than#2

(: ensure the type of $f :) $f as function(xs:integer, xs:integer) as xs:boolean

(: call dynamically $f :) $f(1, 2)

(: all together, this return true for usual my:less-than :) let $f as function(xs:integer, xs:integer) as xs:boolean := my:less-than#2 return $f(1, 2)


summer school


XPath – Inline functions

• function($who) { 'Hello, ' || $who || '!' }returns a function item to an anonymousfunction (AKA an inline function)

• Inline functions allow actually defining functionsdirectly in XPath

• Inline functions introduce closures in XPath• Inline functions, jointly with let, can be used to

create functions in an XPath expression


summer school


XPath – Inline functions

(: looks like an "XPath module" :) let $first := function($arg as xs:string) as xs:string { ... } let $second := function($arg as xs:string) as xs:string { ... $first ... } let $third := function($fun as function(xs:string) as xs:string) as xs:string { ... $fun ... } return ( $third($first), $third($second) )


summer school


XPath – Higher-order functions

• A function item can be passed as parameter toanother function

• A function can return a function item as returnvalue

• A function taking a function item as parameter,or returning a function item as return value, isa higher-order function

• A piece of the algorithm itself of a higher-orderfunction can then be injected by the caller


summer school



(: say there is a function with the following signature in :) (: the static context (in pseudo-XQuery notation here): :) declare function lib:sort( $seq as item()+, $cmp as function(item(), item()) as xs:integer ) as item()+;

(: the usual integer sort can be done with: :) lib:sort( (2, 6, 3, 7, 2, 5, 2, 7, 3, 56, 42), function($lhs as xs:integer, $rhs as xs:integer) as xs:integer { if ( $lhs eq $rhs ) then 0 else if ( $lhs lt $rhs ) then -1 else 1 })


summer school


XPath – HOF exercise

<employee id="jdoe"> <first-name>John</first-name> <last-name>Doe</last-name> <salary>123</salary> </employee>

<employee id="jdoe.1"> <first-name>Jane</first-name> <last-name>Doe</last-name> <salary>234</salary> </employee>

• Implement comparison function for employees• Compare salaries, reverse order• function(item(), item()) as xs:integer


summer school


XPath – HOF exercise

<employee id="jdoe"> <first-name>John</first-name> <last-name>Doe</last-name> <salary>123</salary> </employee>

<employee id="jdoe.1"> <first-name>Jane</first-name> <last-name>Doe</last-name> <salary>234</salary> </employee>

• Implement comparison function for employees• Sort first on last name, second on first name• function(item(), item()) as xs:integer


summer school



(: sorting structured nodes could be done with: :) lib:sort( doc('my-employees.xml')/*/employee, function($lhs as element(), $rhs as element()) as xs:integer { if ( $lhs/last-name lt $rhs/last-name ) then -1 else if ( $lhs/last-name gt $rhs/last-name ) then 1 else if ( $lhs/first-name eq $rhs/first-name ) then 0 else if ( $lhs/first-name lt $rhs/first-name ) then -1 else 1 })


summer school



(: and with a multi-comparator function for composite keys, :) (: that returns a function item composing its parameters: :) lib:sort( doc('my-employees.xml')/*/employee, lib:multi-comparator(( function($p) { $p/last-name }, function($p) { $p/first-name })))


summer school


XPath – Partial application

• Given a function of arity 2, we can create afunction item of arity 1 by binding one of theparameters to a fixed value

• Call the function (or function item) by replacingthe parameters you want to leave blank by “?”

• For instance: compare(?, ?, 'urn:my-collation')• This creates a new function, of arity 2, based

on the standard compare(), but with thecollation param bound to 'urn:my-collation'


summer school


XPath – Partial application

(: say there is a function with the following signature, :) (: to compare 2 items after a custom treatment :) declare function lib:usual-compare( $lhs as item(), $rhs as item(), $access as function(item()) as item() ) as xs:integer;

(: sorting structured nodes could be done with: :) lib:sort( doc('my-employees.xml')/*/employee, lib:usual-compare(?, ?, function($arg as element()) as xs:string { $arg/@id }))



summer school


XQuery 3.1

Even further

summer school


XQuery 3.1

1. Output declaration2. Switch3. Private functions4. Try/catch5. Grouping6. Windowing


summer school


XQuery – Output declaration

• Control the serialization options from withinyour queries

• Like xsl:output• Defined using the options mechanism

declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization";

declare option output:method "xml"; declare option output:encoding "iso-8859-1"; declare option output:indent "yes";


summer school


XQuery – Switch

• New switch/case instruction• Different expressions depending on a value

switch ( $unit ) case 'm' return $meters case 'yd' return $meters * 1.0936 case 'mi' return $meters * 0.00062137119 default return error(xs:QName('app:ERR001'), 'Unknown unit')


summer school


XQuery – Private functions

• Restrict visibility of a function to its module• Defined using the annotations mechanism

(: even if in a library module, it will not be visible : from importing modules :) declare %fn:private function my:convert-to-internal( $param as xs:string ) as xs:string { ... };


summer school


XQuery – Try/catch

declare namespace err = "http://www.w3.org/2005/xqt-errors";

try { 1 div 0 } catch * { $err:code || ': ' || $err:description }

• Catch any error thrown in the “try” clause• Provide information through variables in the

“catch” clause• Can catch specific errors only




summer school


XQuery – Try/catch



• Can have different selective “catch” clauses


try { ... } catch app:ERR001 { 'alternative result' } catch app:ERR002 | app:ERR003 { 42 } catch * { 'what did you do?!?' }


summer school


XQuery – Grouping

• “group by” defines a grouping variable in aFLOWR expression

• The return expression is evaluated once pergroup

• The grouping variable gives the value of thegroup key

• Range variables are bound now to sequences,corresponding to all values of that variable inthat group


summer school


XQuery – Grouping

for $elems in <doc> <elem g="1">A</elem> <elem g="1">B</elem> <elem g="3">C</elem> <elem g="3">D</elem> <elem g="2">E</elem> <elem g="1">F</elem> <elem g="2">G</elem> </doc>/* group by $key := $elems/@g order by $key return <group num="{ $key }">{ $elems/string(.) }</group>

==> <group num="1">A B F</group> <group num="2">E G</group> <group num="3">C D</group>


summer school


XQuery – Windowing

• [Definition: A window is a sequence ofconsecutive items drawn from the bindingsequence.]

• Tumbling windowing “consumes” all adjacentitems of a window at once, then continues withthe next window

• Sliding windowing “consumes” all adjacentitems of a window, then continue with the nextitem (possibly already in the last window)

• Defined by declaring variables and conditions


summer school



for tumbling window $w in <doc> <elem g="1">A</elem> <elem g="1">B</elem> <elem g="3">C</elem> <elem g="3">D</elem> <elem g="2">E</elem> <elem g="1">F</elem> </doc>/* start when true() end $end next $next when $end/@g ne $next/@g return <window group="{ $end/@g }">{ $w/string(.) }</window>

==> <window group="1">A B</window> <window group="3">C D</window> <window group="2">E</window> <window group="1">F</window>


summer school



for sliding window $w in <doc> <elem g="1">A</elem> <elem g="1">B</elem> <elem g="3">C</elem> <elem g="3">D</elem> <elem g="2">E</elem> <elem g="1">F</elem> </doc>/* start when true() end $end next $next when $end/@g ne $next/@g return <window group="{ $end/@g }">{ $w/string(.) }</window>

==> <window group="1">A B</window> <window group="1">B</window> <window group="3">C D</window> <window group="3">D</window> <window group="2">E</window> <window group="1">F</window>



summer school


XSLT 3.0

Package it! Stream it! Recover...

summer school


XSLT 3.0

1. Text value templates2. Generalized patterns3. Mode declaration4. Context item constraint5. Dynamic evaluation of XPath6. Try/catch7. Assertions8. Packages9. Streaming10. Accumulators11. Maps


summer school


XSLT – Text value templates

• Easy text node creation like in XQuery• Use { } to create text nodes in a sequence ctor• Can be enabled/disabled for compatibility• Unlike XQuery, always constructs text nodes

<xsl:template ... expand-text="yes"> <new-school>{ $who }</new-school> <old-school> <xsl:value-of select="$who"/> </old-school> </xsl:template>


summer school


XSLT – Text value templates

<xsl:template name="balance" expand-text="yes"> <xsl:param name="name" as="xs:string"/> <xsl:param name="id" as="xs:string"/> <xsl:param name="amount" as="xs:string"/>

 <xsl:text>Name=</xsl:text> <xsl:value-of select="$name"/> <xsl:text>, id=</xsl:text> <xsl:value-of select="$id"/> <xsl:text>, balance=</xsl:text> <xsl:value-of select="$amount"/> <xsl:text>.</xsl:text> 

Name={ $name }, id={ $id }, balance={ $amount }.

</xsl:template


summer school


XSLT – Generalized patterns

• Template rules can now match any items• They used to be restricted to nodes

<xsl:template ...> <xsl:apply-templates select="'one', 'two', 'three'"/> </xsl:template>

<xsl:template match="~xs:string"> <number>General case: { . }</number> </xsl:template>

<xsl:template match="~xs:string[. eq 'forty two']"> <forty-two>Special treatment.</forty-two> </xsl:template>


summer school


XSLT – Mode declaration

• Modes used to be loosely declared, as soon as atemplate rule declared to be in that mode

• A new mode declaration allows to set propertieson modes:– Visibility (public/private/final, see below)– Default behaviour– Error and warning behaviour– Is it streamable?– Is it usable as an initial mode?– What is the type of the initial context item?


summer school


XSLT – Mode declaration

 <xsl:mode name="my:copy" initial="yes" on-no-match="shallow-copy" visibility="public"/>

<xsl:template match="to-replace" mode="my:copy"> <new-stuf> <xsl:value-of select="..."/> </new-stuf> </xsl:template>


summer school


XSLT – xsl:context-item

• Constraint the type of the context item in anamed template

• Constraint the type of the initial context item– For a specific mode, or for the initial template

<xsl:mode name="db:format"> <xsl:context-item as="document-node(element(db:article))"/> </xsl:mode>

<xsl:template name="something"> <xsl:context-item as="element(stuff)"/> ... </xsl:template>

<xsl:template name="other-thing"> <xsl:context-item use="prohibited"/> ... cannot access the context item ... </xsl:template>


summer school


XSLT – xsl:evaluate

• Allow to dynamically an XPath expressionprovided as a string

• To use with caution, as usual with dynamicevaluation

<xsl:template match="address-book"> <xsl:param name="sort-expr" select=" 'first-name' "/> <xsl:for-each select="contact"> <xsl:sort>   <xsl:evaluate xpath="$sort-expr" context-item="."/> </xsl:sort> </xsl:for-each> </xsl:template>


summer school


XSLT – Try/catch

• Catch any error thrown in the “try” clause• Provide information through variables in the

“catch” clause (shared with XQuery)• Can catch specific errors only

<xsl:stylesheet ... xmlns:err="http://www.w3.org/2005/xqt-errors">

<xsl:try select="1 div 0"> <xsl:catch select="$err:code || ': ' || $err:description"/> </xsl:try>


summer school


XSLT – Try/catch

• Can have different selective “catch” clauses

<xsl:stylesheet ... xmlns:app="http://example.org/ns/my-app" xmlns:err="http://www.w3.org/2005/xqt-errors">

<xsl:try> <sequence-ctor> ... </sequence-ctor> <xsl:catch errors="app:some-error" select="'alternative res'"/> <xsl:catch errors="app:foo app:bar"> <boom> <xsl:value-of select="$err:description"/> </boom> </xsl:catch> <xsl:catch errors="*" select="'what did you do?!?'"/> </xsl:try>


summer school


XSLT – Assertions

• Allow writing assertions directly in XSLT• Assertion checking can be disabled• If the assertion fails, a dynamic error is thrown

 <xsl:param name="DEBUG" select="false()" static="yes"/>

<xsl:function name="my:convert" as="xs:integer"> <xsl:param name="numer" as="xs:integer"/> <xsl:param name="denom" as="xs:integer"/> <xsl:assert test="$denom ne 0" use-when="$DEBUG"/> <xsl:sequence select="$numer div $denom"/> </xsl:function>


summer school


XSLT – Packages

• Higher level of separation than xsl:stylesheet• Packages are comprised of stylesheets• Isolate better dependencies• Support processors for separate compilation• Provide developers with real encapsulation

mechanism• In my humble opinion the necessary step

towards really reusable and deliverable XSLTlibraries

• Might support XQuery


summer school


XSLT – Packages

• Clear borders between “imports” (called “usepackage”)

• Opportunity to control what components can beoverridden by “using packages”

• Opportunity to control what components areactually imported from a “used package”


summer school


XSLT – Packages

<xsl:package xmlns:lib="http://example.org/lib/package" xmlns:iface="http://example.org/lib/package/iface" xmlns:impl="http://example.org/lib/package/impl" name="http://example.org/lib/package" package-version="1.2" version="3.0"> <xsl:stylesheet version="3.0"> <xsl:function name="lib:..."> ... </xsl:function> </xsl:stylesheet> <xsl:expose component="function" names="lib:*" visibility="final"/> <xsl:expose component="function" names="iface:*" visibility="public"/> <xsl:expose component="function" names="impl:*" visibility="private"/> </xsl:package>


summer school


XSLT – Packages

<xsl:package xmlns:lib="http://example.org/lib/package" name="http://h2oconsulting.be/xyz-project/my-package" package-version="0.1" version="3.0"> <xsl:use-package name="http://example.org/lib/package" package-version="1.2"> <xsl:accept component="function" names="lib:*" visibility="private"/> </xsl:use-package> <xsl:stylesheet version="3.0"> ... </xsl:stylesheet> </xsl:package>


http://example.org/lib/package

summer school


XSLT – Streaming

• The XSLT Working Group “defines” streamingas “The term streaming refers to a manner ofprocessing in which documents (such as sourceand result documents) are not represented by acomplete tree of nodes occupying memoryproportional to document size, but instead areprocessed "on the fly" as a sequence of events,similar in concept to the stream of eventsnotified by an XML parser to represent markupin lexical XML”

• ...


summer school


XSLT – Streaming

• Basically, streaming does not require bufferingthe input in memory (or not much)

• Streaming does not require buffering the outputin memory (or not much)

• Streaming in XSLT 1.0 was thought to beachievable by optimization

• Streaming support in XSLT 3.0 is done byproviding the processor with more informationit could not easily infer

• The developer does know better, usually


summer school


XSLT – Streaming

• The basic idea is you can navigate the inputonly on the way forward

• You can access nodes only once (even of coursein predicates or any other expressions, e.g.used in a condition)

• The name of the ancestor elements, and theirattributes, are also available

• At any point during the processing, your areeither in streaming mode or not

• When in streaming mode, the processorenforces you are not doing anything forbidden


summer school


XSLT – Streaming

<xsl:mode name="s" streamable="yes"/>

<xsl:template name="main"> <xsl:stream href="book.xml"> <xsl:apply-templates select="book" mode="s"/> </xsl:stream> </xsl:template>

<xsl:template match="book" mode="s"> <xsl:apply-templates select="chapter" mode="s"/> </xsl:template>

<xsl:template match="chapter" mode="s"> <chapter>   <xsl:copy-of select="title | para"/> </chapter> </xsl:template>


summer school


XSLT – Accumulators

• First been design to support accumulatingvalues even in streaming mode

• Convenient for any accumulation of values• Based on pattern matching• An accumulator associates 2 values to each

node in a tree– The pre-descent value– And the post-descent value

• Use fn:accumulator-before() to access the pre-descent value (resp. fn:accumulator-after())


summer school



• An accumulator has an initial value• Then the value can be changed when matching

another node• Document is walked in document order• Each value can rely on the previous value• For instance, in a book, numbering figures

starting at 1 again in every chapter• Value accessible via a new function (named

after the accumulator)


summer school



<person> <name> <firstName>Florent</firstName> <middle>Emile Christian Ghislain Etienne</middle> <lastName>Georges</lastName> </name> <phone> <mobile>+32 497 113 244</mobile> </phone> </person>

• Reminder: document order


summer school



<xsl:accumulator name="f:figure-nr" as="xs:integer" initial-value="0" streamable="yes"> <xsl:accumulator-rule match="chapter" select="0"/> <xsl:accumulator-rule match="figure" select="$value + 1"/> </xsl:accumulator> <xsl:mode streamable="yes"/>

<xsl:template match="figure"> <xsl:apply-templates/> <xsl:text>Figure </xsl:text> <xsl:value-of select="accumulator-after('f:figure-nr')"/> </xsl:template>


summer school


XSLT – Arrays

• Sequences are flat• Maps allow to represent “nested sequences”

without the copy semantics of nodes• As everything else in XDM, arrays are non

mutable objects• An array is an ordered list of values• A value is any arbitrary sequence• An array is a function• An array can be seen as a map from integers to

values


summer school


XSLT – Arrays

<xsl:variable name="v1" as="array(item()*)" select=" array { 'bleu', (1, 2, 3), (), *[@color eq 'red'] }"/>

<xsl:variable name="v2" as="array(item()*)" select=" [ 'bleu', (1, 2, 3), (), *[@color eq 'red'] ]"/>

<xsl:sequence select="$v1(3)"/>

<xsl:sequence select="$v2(3)"/>


summer school


XSLT – Maps

• AKA hash dictionaries• Sequences are flat• Maps allow to represent “structured sequences”

without the copy semantics of nodes• As everything else in XDM, maps are non

mutable objects• A map is a set of pairs “key / value”• A key is an arbitrary atomic value (such as

xs:string, xs:integer, etc.)• A value is any arbitrary sequence• A map is a function


summer school


XSLT – Maps

<xsl:variable name="v1" as="map(xs:string, item()*)" select=" map { 'blue' : 'bleu', 'green' : (1, 2, 3), 'red' : *[@color eq 'red'] }"/>

<xsl:variable name="v2" as="map(xs:string, item()*)" select=" map:merge(( map:entry('blue', 'bleu'), map:entry('green', (1, 2, 3)), map:entry('red', *[@color eq 'red']) ))"/>

<xsl:sequence select="$v1('red')"/>

<xsl:sequence select="map:keys($v1)"/>

<xsl:sequence select="map:get($v2, 'blue')"/>

<xsl:sequence select="map:remove($v2, 'green')"/>



summer school


Questions

And answers, hopefully

summer school


Questions

• Any question?• For further reference:

– http://w3.org/TR/xpath-3/– http://w3.org/TR/xpath-functions-3/– http://w3.org/TR/xquery-3/– http://w3.org/TR/xslt-30/


http://w3.org/TR/xpath-3/

http://w3.org/TR/xpath-functions-3/

http://w3.org/TR/xquery-3/

http://w3.org/TR/xslt-30/

Documents

XSLT and XQuery - fgeorges.orgfgeorges.org/papers/fgeorges-xmlss-xslt-trends-2016.pdfIntroduction • XPath 2.0, XSLT 2.0 and XQuery 1.0 have been released in 2007 • XPath and XSLT