INNOV-12: Transforming Non ‑ XML Documents with XML Tools Tony Lavinio Principal Software Architect, Stylus Studio

2© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Let’s use Stylus Studio! n eWEEK 5 th Annual Excellence Awards Finalist n Integrated with: –Saxon and 8.4 from Saxonica –Xalan-J from the Apache Xerces project –MSXML 3 and 4 –System.XML XSLT from.Net –Sleepycat Berkeley DB XML –Mark Logic XDMS It’s what all the cool people are using

3© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Build on Standards n XSLT 1.0 and 2.0 n XQuery n XPath 1.0 and 2.0 n DTD n W3C XML Schema n SQL/XML n JDBC and ODBC n SOAP, WSDL, UDDI n OASIS Catalogs n Java™ n JSP n JAXP (formerly TrAX) n JAXB n HTTP[S] n FTP n [X]HTML n EDIFACT/CEFACT n X12 Everything-but-the-kitchen sink

4© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools XML or Not XML, Who Cares? n XQuery and XSLT only act on XML –CSV, EDI, etc. are not XML –What can we do? n Batch convert? — No! n On-the-fly conversions –Convert-to-XML feature –Native adapters Stylus Studio can treat non-XML data as XML

5© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Agenda n Convert-to-XML feature n The Architecture n XQuery and XSLT on non-XML n Deploying n Running

6© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Convert-to-XML User Interface Maps flat input data to XML output data Properties Pane XML Output Preview Input Canvas Schema Pane

7© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Convert-to-XML Features n Maps text or binary formats –CSV, TSV, EDI — even xBase n Gobs of encodings n Fixed or variable-length fields n Variety of data types supported n Regular expression matching n Streaming engine supports large input files

8© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Demonstration of Convert-to-XML UI n Let’s build a CSV converter! n Let’s build an EDI converter! (Sample of X12 EDI data)

9© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Agenda n Convert-to-XML feature n The Architecture n XQuery and XSLT on non-XML n Deploying n Running

10© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Extensible File System Support n Traditional file systems –file:, ftp: and http[s]: (WebDAV) schemes n Web Service Call Composer –wscc: scheme n SQL/XML for Relational Databases –db: scheme n Sleepycat Berkeley DB XML –dbxml: scheme Lets us treat any type of resource as a URI

11© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools The Adapter URL Syntax Two forms: n Convert-to-XML: adapter:map.conv?url n Native XML Adapters: adapter:name:[option=value:…]?url

12© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Implementation n Convert-to-XML has two parts –User interface –Runtime engine n The runtime engine is just one of many native converters that can be plugged into the Adapter File System. n Designed as a streaming engine for low footprint What’s under the hood

13© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Adapter File System It plugs in – and supports its own plug-ins Stylus Studio file: http: and https: (WebDAV) db: custom file system driver java: user: adapter: CtoXMLCSVdBase.d *.txt *.csv *.html *.xslt *.??? *.d

14© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Agenda n Convert-to-XML feature n The Architecture n XQuery and XSLT on non-XML n Deploying n Running

15© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools A Simple Confluence of Ideas +If we combine the idea that adapters convert non-XML to XML, +With the idea that adapters can be addressed with URLs, + With the idea that XQuery and XSLT work on URL-addressable XML, = We have an integration tool > 3

16© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Design XSLT or XQuery n Build an XSLT transformation n Input is EDI n Desired output is CSV Take the non-XML and manipulate it as XML

17© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Agenda n Convert-to-XML feature n The Architecture n XQuery and XSLT on non-XML n Deploying n Running

18© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Assembling the Building Blocks Here’s what we’ve done: 1.Used the Convert-to-XML map we built earlier for input 2.Used a CSV file via adapter as the target 3.Built XSLT from and to representative non-XML documents And here’s what we’re about to do: 1.Build a Java program to call XSLT 2.Call the Java program from the 4GL

19© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools The Runtime n We use Saxon or 8.4, or Xalan-J as the XSLT engine, or Saxon 8.4 as the XQuery engine n The Stylus Studio runtime adds the missing pieces, such as our URL resolver and adapter libraries n We’ll even generate the code for you

20© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Calling Our Transform // This is the “before” version… String xslURL = "file:///c:/temp/EXCHANGE.xsl"; String xmlURL = "file:///c:/temp/EXCHANGE.xml"; Source xsl = new StreamSource(new URL(xslURL).openStream(), xslURL); Source xml = new StreamSource(new URL(xmlURL).openStream(), xmlURL); Result out = new StreamResult(System.out); TransformerFactory factory = TransformerFactory.newInstance(); Transformer t = factory.newTransformer(xsl); t.transform(xml, out); This is just plain standard JAXP code

21© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Same, But with Stylus URL Resolver System.setProperty(STYLUS_ROOTDIR, "."); System.setProperty(STYLUS_APPDATA, "."); String xslURL = "file:///c:/temp/EXCHANGE.xsl"; String xmlURL = "adapter:EXCHANGE.conv?file:///c:/temp/EXCHANGE.edi"; Source xsl = StylusFileFactory.getFactory().resolve(xslURL, null); Source xml = StylusFileFactory.getFactory().resolve(xmlURL, null); Result out = new StreamResult(System.out); TransformerFactory factory = TransformerFactory.newInstance(); Transformer t = factory.newTransformer(xsl); t.transform(xml, out); Add two property settings, and use the new factory

22© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools URI Resolvers are for Writing Too Since adapters can go either way or go both ways, so can the URI resolver. Non-XML to XMLBoth WaysXML to Non-XML

23© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Agenda n Convert-to-XML feature n The Architecture n XQuery and XSLT on non-XML n Deploying n Running

24© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Finishing up… n Compile the.java n Move things into the proper locations n Set up the 4GL n Go for it!

25© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools EDI  XML  XSLT  CSV  4GL + = Let’s See It All Work Together

26© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Resources n Stylus Studio Web Site n Stylus Scoop Newsletter n The W3C XSLT and XPath Standards n The W3C XQuery Standard

27© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools In Summary n Stylus Studio lets you treat non-XML data sources as XML n Nothing beats XSLT/XQuery for transforming XML n The XSLT/XQuery Mappers make standards-based integration simpler

28© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Questions?

29© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools Thank you for your time!

30© 2005 Progress Software Corporation INNOV-12 Transforming Non-XML Documents with XML Tools