Efficient XML Aaron Braeckel Briefing to FAA Program Management 28 Dec 2008 National Center for Atmospheric Research Boulder, CO Copyright © 2008-2009.

Slides:

Advertisements

Similar presentations

Microsoft Office System UK Developers Conference Radisson Edwardian, Heathrow 29 th & 30 th June 2005.

Advertisements

Performance Evaluation of the CPE WAN Management Protocol Serafeim Papastefanos NTUA.

Application of XSLT Stylesheet to XML Atsushi Shimazaki Japan Meteorological Agency - WMO Workshop on Metadata - September 2005, Beijing, China.

Delivering Digital Services NextGen Web Service Implementation by the National Weather Service Ryan Solomon Project Manager NOAA NextGen IT Web Services.

JSI Sensor Middleware. Slide 2 of x Embedded vs. Midleware based Architecture for Sensor Metadata Management Embedded approach assign an IP address to.

Open-DIS and XML DIS in Other Formats. Distributed Interactive Simulation DIS is an IEEE standard for simulations, primarily virtual worlds Binary protocol:

NextGen Network-Enabled Weather (NNEW) Concepts Aaron Braeckel.

WXXM 2.0 Delivering Digital Services Presented By:

© 2010 Microsoft Corporation. All rights reserved. Quality Assurance: Towards Tools for Characterizing and Comparing Digital Documents Natasa Milic-Frayling.

Philips Research France Delivery Context in MPEG-21 Sylvain Devillers Philips Research France Anthony Vetro Mitsubishi Electric Research Laboratories.

XML Introduction What is XML –XML is the eXtensible Markup Language –Became a W3C Recommendation in 1998 –Tag-based syntax, like HTML –You get to make.

BinX and Astronomy Bob Mann Institute for Astronomy and National e-Science Centre.

Efficient XML Interchange. XML Why is XML good? A widely accepted standard for data representation Fairly simple format Flexible It’s not used by everyone,

What is JPEG 2000 Robert Han ECE533 Final Project Presentation University of Wisconsin - Madison.

Learning Resource iNterchange

The British Library’s METS Experience The Cost of METS Carl Wilson

OCLC Online Computer Library Center Two Paths to Interoperable Metadata Jean Godby, Devon Smith, Eric Childress DC-2003 September 29, 2003.

By: Shawn Li. OUTLINE XML Definition HTML vs. XML Advantage of XML Facts Utilization SAX Definition DOM Definition History Comparison between SAX and.

HTTP-NG Briefing at Internet PSIG meeting, 8 June 1998; Mike Spreitzer, Xerox PARC 1 HTTP - Next Generation Mike Spreitzer Xerox PARC CSL 8 June 1998.

EARTH SCIENCE MARKUP LANGUAGE “Define Once Use Anywhere” INFORMATION TECHNOLOGY AND SYSTEMS CENTER UNIVERSITY OF ALABAMA IN HUNTSVILLE.

Sustainability Workshop Day 1 November 30’07 PLANETS Products and Discussion of Sustainable Models.

Introduction to XML cs3505. References –I got most of this presentation from this site –O’reilly tutorials.

Worshipping at the Shrine: Myths and Legends from comp.text.xml Kerry “the heretic” Raymond, CiTR.

XML The Overview. Three Key Questions What is XML? What Problems does it solve? Where and how is it used?

A In-Memory Compressed XML Representation of Astronomical Data PPARC UK e-Science Postgraduate School ’05 O’Neil Delpratt – PhD Student University of Leicester.

Java-Based Middleware IT 490 Stan Senesy IT Program NJIT.

Intro. to XML & XML DB Bun Yue Professor, CS/CIS UHCL.

The future of MINC Robert D. Vincent

Lecture Set 12 Sequential Files and Structures Part C – Reading and Writing Binary Files.

EARTH SCIENCE MARKUP LANGUAGE Why do you need it? How can it help you? INFORMATION TECHNOLOGY AND SYSTEMS CENTER UNIVERSITY OF ALABAMA IN HUNTSVILLE.

Ontologies and Lexical Semantic Networks, Their Editing and Browsing Pavel Smrž and Martin Povolný Faculty of Informatics,

XML Data Storage Joe Carroll Russell Gibbons. Agenda What is XML Storage of XML Benefits of XML Databases Problems with XML Databases Discussion.

File format registries - a global infrastructure for local persistence Andreas Aschenbrenner, ERPANET.

XML The “E-Lance Economy” or “Digital Economy” is a new challenge for interacting over networks. XML was developed by the World Wide Web Consortium (W3C)

EXI Comparisions. EXI Emerging W3C standard, now in “final call” status on the standards track Provides a more efficient, alternate.

Huffman coding Content 1 Encoding and decoding messages Fixed-length coding Variable-length coding 2 Huffman coding.

Week 11: Open standards and XML MIS 3537: Internet and Supply Chains Prof. Sunil Wattal.

CCGrid 2014 Improving I/O Throughput of Scientific Applications using Transparent Parallel Compression Tekin Bicer, Jian Yin and Gagan Agrawal Ohio State.

Efficient XML Interchange High Performance XML Don McGregor (mcgredo (at) nps.edu) Don Brutzman (brutzman (at) nps.edu)

XML and Its Applications Ben Y. Zhao, CS294-7 Spring 1999.

AxKit A member of the Apache XML project Ryan Maslyn Kyle Bechtel.

XML eXtensible Markup Language. XML A method of defining a format for exchanging documents and data. –Allows one to define a dialect of XML –A library.

1 TOPIC 6 DATABASE 6.1 Introduction to Database 6.2 Basic Concept of Database 6.3 Database Object DATABASE.

Performance of Compressed Inverted Indexes. Reasons for Compression  Compression reduces the size of the index  Compression can increase the performance.

Dom and XSLT Dom – document object model DOM – collection of nodes in a tree.

PRESENTATION DAY Group ID: gp09-cmg Speaker: Matthew Albers RFID APPLICATION DEVELOPMENT SUITE.

Improvement of Schema-Informed XML Binary Encoding Using Schema Optimization Method BumSuk Jang and Young-guk Ha' Konkuk University, Department of Computer.

©Silberschatz, Korth and Sudarshan10.1Database System Concepts W3C - The World Wide Web Consortium W3C - The World Wide Web Consortium.

Comp 335 File Structures Data Compression. Why Study Data Compression? Conserves storage space Files can be transmitted faster because there are less.

Delivering Digital Services US Weather Standards and Codes Presented By: Matt Peroutka Aaron Braeckel Steve Olson Date:August 28, 2014.

Standards for representing meeting metadata and annotations in meeting databases Standards for representing meeting metadata and annotations in meeting.

Steps to integrate XML How does XML processing work? Simple uses of passive DOM objects Adding behaviour to information A converter and translator subsystem.

Angela McCarthy CP5080, SP  Received: 14 August 2008  Revised: 13 November 2008  Written by Sherif Sakr of University of New South Wales, Australia.

XML The Overview. Three Key Questions What is XML? What Problems does it solve? Where and how is it used?

Data Format Description Language (DFDL) WG Martin Westhead EPCC, University of Edinburgh

Challenges in XML It’s good… but is it good enough? Siddhesh Bhobe Persistent eBusiness Solutions.

PRESENTATION ON SURVEY OF RESEARCH Research Topic ( MIGRATION OF WEB SERVICE TO MOBILE AND WIRELESS APPLICATION ‘S) GUIDED BY: PRESENTED BY: MISS. UMA.

I Copyright © 2004, Oracle. All rights reserved. Introduction.

1 Introduction to XML Babak Esfandiari. 2 What is XML? introduced by W3C in 98 Stands for eXtensible Markup Language it is more general than HTML, but.

XBRL-CSV Overview.

Compression & Huffman Codes

Building A Repository for Digital Objects

A brief overview of the contribution

Session I - Introduction

Session I - Introduction

The Linux Command Line Chapter 18

XML Problems and Solutions

Using NFFI Web Services on the tactical level: An evaluation of compression techniques 13th ICCRTS: C2 for Complex Endeavors Frank T. Johnsen.

OPeNDAP/Hyrax Interfaces

Presentation transcript:

Efficient XML Aaron Braeckel Briefing to FAA Program Management 28 Dec 2008 National Center for Atmospheric Research Boulder, CO Copyright © University Corporation for Atmospheric Research

2 NextGen Network Enabled Weather (NNEW) Background and Motivation Extensible Flexible Human-readable Validatable Can be transformed Namespaced … XML Benefits: Copyright © University Corporation for Atmospheric Research

3 NextGen Network Enabled Weather (NNEW) Background and Motivation XML Issues: Copyright © University Corporation for Atmospheric Research One of the unique (and difficult) weather problems is data volume Processing efficiency Encoding time Decoding time Message throughput Battery life (handheld devices, etc) Compactness Storage Archival Bandwidth/transmission Has a (non-trivial) recurring cost in some cases Next-generation aircraft Handheld devices

4 NextGen Network Enabled Weather (NNEW) Background and Motivation XML Assessments: Copyright © University Corporation for Atmospheric Research DoD Comparison (binary to XML) 1 10x, 17x, 22x, and 121x larger with XML W3C XML Binary Characterization Working Group Collected requirements W3C Efficient XML Interchange Working Group Performed measurements Made recommendations NCAR Preliminary Efficient XML Assessment Compared Fast Infoset vs normal XML representations Fast Infoset 75% of XML size Fast Infoset 33% of parsing time NCAR Efficient XML Assessment Measurements and recommendation Will likely use the EXI measurement framework WXXM 1 Efficient XML – Taking Net-Centric Operations to the Edge. John Schneider

5 NextGen Network Enabled Weather (NNEW) Background and Motivation W3C Binary Characterization: Copyright © University Corporation for Atmospheric Research Collected use cases Collected desirable features of different approaches Listed a minimum set of features for a recommended solution “Binary XML is needed” W3C Efficient XML Interchange: Created a measurement framework Collected a varied set of XML files for different use cases Made measurements of different solutions Recommended a solution Produced a specification (EXI) “the results indicate it is possible to achieve substantial gains over XML… in a wide variety of use cases” 3

6 NextGen Network Enabled Weather (NNEW) Solutions Basic Approaches: Copyright © University Corporation for Atmospheric Research Compression (GZIP, ZIP, TAR, etc) Worse processing efficiency, does not help with the characters -> value processing issue Must be decompressed before being operated upon (XSLT) XML-wrapped binary chunks XML for metadata or other selected portions Binary portions are “opaque”, no XML compatibility Efficient/binary XML formats Encode XML documents in a different form

7 NextGen Network Enabled Weather (NNEW) Solutions Recommendation: Copyright © University Corporation for Atmospheric Research EXI format Last Call Working Draft Likely to be the general industry standard for efficient XML transmission* Open source libraries (but still evolving) OpenEXI* Exificient … Advantages: XML flexibility XML tooling support (when developed) Increased efficiency Lossless conversion to/from XML Disadvantages: Not human readable (without conversion) Use SAX or StAX-based parsing techniques DOM has a non-trivial overhead in some cases (such as memory usage)

8 NextGen Network Enabled Weather (NNEW) Exificient/GZIP experiment Copyright © University Corporation for Atmospheric Research GZIP: 0.07 of original file size EXI: 0.13 of original file size

9 NextGen Network Enabled Weather (NNEW) Fast Infoset to XML comparison Copyright © University Corporation for Atmospheric Research File sizeParsing time AIRSIGMETs XML (5 records) 7kb18ms AIRSIGMETs FI3kb (0.43)13ms (0.72) METARs XML (1481 records) 1167kb84ms METARs FI373kb (0.32)56ms (0.667) PIREPs XML (158 records) 155kb29ms PIREPs FI51kb (0.33)29ms (1.0) TAFs XML (177 records)471kb57ms TAFs FI98kb (0.208)39ms (0.684)