Format Independent Change Detection & Propagation (FCDP) in Support of Mobile Computing Michael Lanham, Ajay Kang, Joachim Hammer, Abdelsalam Helal, Joseph Wilson

8 Oct 022 of 22 Agenda Motivation & Problem Statement Related Research FCDP Implementation Experimental Results Future Research Conclusions Questions

8 Oct 023 of 22 Sponsors Research sponsored in part by the National Science Foundation through grant number OR Research sponsored in part by the United States Military Academy Part of Ubiquitous Data Access (UbiData) Project –Lead Investigators: Dr. Helal and Dr. Hammer –Various Ph.D. and Masters Candidates

8 Oct 024 of 22 Motivation vdiff

8 Oct 025 of 22 Problem Statement How to identify meaningful changes between two XML documents that do not have identical element tag and attribute sets? How to apply those changes on a copy of the source XML document?

8 Oct 026 of 22 Related Research (1 of 3) Bandwidth Adaptation –Puppeteer [Rice U] Dynamic adaptation to network conditions (MS Office Only) MS-OLE DOM and Puppeteer Intermediate Form –Odyssey [CMU] Actively adapts to network conditions Requires modification of applications –Neither support disconnected operations

8 Oct 027 of 22 Related Research (2 of 3) Text- and Byte-based Difference Detection –(line based) diff (GNU diff) –Xdelta: binary delta based file system from UCB –Rsync: allows binary diff on data on separate machines (None can support B/W adaptation or cross-application diff/patch)

8 Oct 028 of 22 Related Research (3 of 3) XML specific algorithms –Sun [diffmk], IBM Alphaworks [XMLdiff, and treediff ] Open API, closed source, no ‘move’ support for IBM, no ‘update’ for either –UMD [laDiff], and INRIA Rocquencourt, France [XyDiff] Open source, Supports ‘move’ and ‘update’ XyDiff uses XIDs (aka eXternal IDentifiers) (Neither can cope with non-identical tag/attribute sets)

8 Oct 029 of 22 FCDP Design Goals Minimize connect time and bandwidth requirements when in a mobile mode Application transparent bandwidth adaptation Application transparent data accessibility Cross-Application difference detection

8 Oct 0210 of 22 Architectural Overview Laptop Mobile Device M-Mem Specific Format Document XML Converter Limited Content Doc Request for import including Target App Palm Mobile Device M-Mem Specific Format Doc Publish Document XML Encoded Data FCDP Server F-Mem Application and B/W filtered XML / Target App document XSLT Convertor Custom Splitter Target App Conversion Rules Avail Bandwidth Sensor Target App Sensor

8 Oct 0211 of 22 FCDP Server F-Mem Mobile Device M-Mem Text (ver 1(-)’) to AbiWord XML(-) Converter XML vdiff util XML delta script XML deltaApply util AbiWord XML Doc v1 XML delta script Text Document v0(-) AbiWord XML to Text Converter Text Doc v0(-) AbiWord Generated XML Doc v 0 Text Document Text patch script Text Doc v1(-) Text diff util patch util XML  txt  txt’  XML(-)

8 Oct 0212 of 22 Content Conversion Management v0 Document Tag 1 Tag n Tag 4 v1 Document Tag 2 Tag n Tag 3 Intersection Map Symmetric Difference Map Lossless Convert or Lossy? If Lossy, must track omitted data Track via Intersection or Symmetric Difference Maps

8 Oct 0213 of 22 vdiff() 15 line algorithm (specifics in the paper) Of 13 modules inherited from INRIA’s XyDiff, we have modified 5 and added 2 more. –The modifications included complete rewrites of PeepHoleOptimization –Brand new modules for parsing the StructuralMapInfo and when we need to AdjustForUnSharedChildren –Entire project’s code base now sits at a little more than 16K lines, XyDiff was just over 8.6K lines

8 Oct 0214 of 22 Generation II vdiff() Match text nodes using LCS algorithms and partial string matching to create as many text node matchings as possible Adjust structure of v1minusDOM so it is isomorphic to v0 Optimize Matches Construct Delta Script

8 Oct 0215 of 22 Experimental Setup XML document version n Text document version n Text 2 AbiW Converter 0-50% of version n’s paragraph insert, delete, move, update 10-40% of paragraph content Text document version n+i Primitive XML document version n+i Diff script DiffApplyProgram Diff script DiffApplyProgram XML document version n+1 AbiWord Statistics Capture, Parse, and Analysis XyDiff vdiff / vdiff2()

8 Oct 0216 of 22 Experimental Results: ASCII (v0(-)) vs AbiWord (v0) in the TestSet

8 Oct 0217 of 22 Experimental Results: Diff versus Value Shipping Text Documents

8 Oct 0218 of 22 Experimental Results: Cumulative Diff Script Sizes After 960 Test Cases

8 Oct 0219 of 22 Experimental Results: Missing node counts for vdiff2() and XyDiff against the reference modified documents

8 Oct 0220 of 22 Future Research (1 of 2) Peephole Optimization- –Customized –Q-grams –Partial String Matching Child Reordering –Solve with placeholders? –Use stylized markup in text only docs as placeholder –Non-Provably correct otherwise Expand classes of XML documents and applications supported for further logic / implementation verification

8 Oct 0221 of 22 Conclusions We have improved current state of the art in mobile computing in the following manner: –Content transformation –Content reduction –Change detection between reduced & transformed documents and canonical XML documents –Set stage for expansion and refinements of above

8 Oct 0222 of 22 Questions