1. The Database and Info. Systems Lab. University of Illinois at Urbana-Champaign Context-Aware Wrapping: Synchronized Data Extraction Shui-Lung Chuang, Kevin C. Chang, ChengXiang Zhai

2. Structured Data on the Web Source amazon Data in less-structured format query

3. Structured Data on the Web Source amazon Data in less-structured format Structured Data Wrapper [$title] by [$author] ($fmt $pdate) …. [$price] querywrapping C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 …

4. Automatic Wrapper Construction Source amazon Data in less-structured format Structured Data Wrapper Effective Java PL Guide by J. Bloch ( Paperback Jun 2001 ) $44.99 C Programming Lang.. by B. Kernighan et al ( Mar 1988 ) $44.20 Data Page(s): Automatic Wrapper Construction Discover Template (E.g., RoadRunner (VLDB’01), ExAlg (SIGMOD’03)) variant parts are data slots querywrapping C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 … [$title] by [$author] ($fmt $pdate) …. [$price]

5. Problem I: Correct Inherent Errors? Source Data in less-structured format Structured Data amazon Wrapper Data Page(s): Automatic Wrapper Construction (E.g., RoadRunner (VLDB’01), ExAlg (SIGMOD’03)) [$title] by [$author] ($fmt $pdate) …. [$price] Effective Java PL Guide by J. Bloch ( Paperback Jun 2001 ) $44.99 C Programming Lang.. by B. Kernighan et al ( Mar 1988 ) $44.20 Data Page(s): Discover Template variant parts are data slots C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 …  data fields format pub-date querywrapping

6. Putting In Integration — Isolated Wrapping Wrapper am amazon … … … …… … … … bn.com borders Wrapper bn Wrapper pw C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) …… Britchie. Kernighan Biarne Stroustrup … ……………… Price: $39.48 Price: $51.99 … Structured DataSources in a domain C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 … The Art of Comp. Prog. Modern Operating Systems … … … D. Knuth A. Tanenbaum … … … ……………… Hardcover $16.38 Paperback $6.95 …

7. Problem II: Achieve Consistency? Wrapper am amazon … … … …… … … … bn.com borders Wrapper bn Wrapper pw C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) …… Britchie. Kernighan Biarne Stroustrup … ……………… Price: $39.48 Price: $51.99 … Structured DataSources in a domain C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 … Inconsistency The Art of Comp. Prog. Modern Operating Systems … … … D. Knuth A. Tanenbaum … … … ……………… Hardcover $16.38 Paperback $6.95 …

8. Scenario 1: Can we leverage domain knowledge? Leverage the users’ efforts C++ Programming Language (3rd Ed.) Bjarne Stroustrup Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 List Price: $43.95 Min Price: $8.95 ………Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 … … … List Price: $43.95 Min Price: $8.95 Automatic wrapping

9. Scenario 1: Can we leverage domain knowledge? Leverage the users’ efforts Domain Knowledge ISBN: Price: Date: “ISBN:” + 10 digits “? Price:” + $ Mon Year Year: 4 digits Mon: Jan | Feb | ….. … … … ………Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 … … … List Price: $43.95 Min Price: $8.95 ………Addison-WesleyISBN: 0201539926 Pub Date: Jul 1991 List Price: $43.95Min Price: $8.95 ISBN:Date:Price: Automatic wrapping C++ Programming Language (3rd Ed.) Bjarne Stroustrup Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 List Price: $43.95 Min Price: $8.95

10. Scenario 2: Can we leverage Existing Wrappers? Leverage the previous efforts The Art of Comp. Prog. Modern Operating Systems … … … … … D. Knuth A. Tanenbaum … Hardcover $143.65 Paperback $45.99 … … … Amazon Existing wrapped sources: New sources to wrap: C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Paperback … … … Mar 1988 Jun 2001 … … … ……………… $44.20 $44.99 … Correct Wrapper am BN.com

11. Scenario 2: Can we leverage Existing Wrappers? Leverage the previous efforts The Art of Comp. Prog. Modern Operating Systems … … … … … D. Knuth A. Tanenbaum … Hardcover Paperback … … … $143.65 $45.99 … Existing wrapped sources: New sources to wrap: C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Paperback … … … Mar 1988 Jun 2001 … … … ……………… $44.20 $44.99 … Correct Wrapper am BN.com Amazon

12. Scenario 3: Can we leverage Other Sources in integration? Leverage the concurrent efforts The Da Vinci Code Lord of the Rigns … … … … … Hanegraaff & Maier Tolkien … …… UK Edition 1st Edition … 5.29 EUR 6.38 EUR … C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 … New sources to wrap: C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) …… Britchie. Kernighan Biarne Stroustrup … ……………… Price: $39.48 Price: $51.99 … The Art of Comp. Prog. Modern Operating Systems … … … … … D. Knuth A. Tanenbaum … Hardcover $143.65 Paperback $45.99 … … … Amazon BN.com Borders Powells

13. Scenario 3: Can we leverage Other Sources in integration? Leverage the concurrent efforts The Da Vinci Code Lord of the Rigns … … … … … Hanegraaff & Maier Tolkien … …… UK Edition 1st Edition … 5.29 EUR 6.38 EUR … C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 … New sources to wrap: C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) …… Britchie. Kernighan Biarne Stroustrup … ……………… Price: $39.48 Price: $51.99 … The Art of Comp. Prog. Modern Operating Systems … … … … … D. Knuth A. Tanenbaum … Hardcover $143.65 Paperback $45.99 … … … Amazon BN.com Borders Powells

14. Scenario 3: Can we leverage Other Sources in integration? Leverage the concurrent efforts The Da Vinci Code Lord of the Rigns … … … … … Hanegraaff & Maier Tolkien … …… UK Edition 1st Edition … 5.29 EUR 6.38 EUR … C Programming Lang. Effective Java PL guide … … … B. Kernighan et al J. Bloch … … … Mar 1988 Paperback Jun 2001 … … … … … ……………… $44.20 $44.99 … New sources to wrap: C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) …… Britchie. Kernighan Biarne Stroustrup … ……………… Price: $39.48 Price: $51.99 … The Art of Comp. Prog. Modern Operating Systems … … … … … D. Knuth A. Tanenbaum … Hardcover Paperback … … … $143.65 $45.99 … Amazon BN.com Borders Powells

15. Our Approach: Context-Aware Wrapping Wrapping should exploit its context Context-Aware Wrapping Domain Knowledge ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: New sources: Amazon buy.com borders powells AnotherBook.com The Context of AnotherBook.com … … … … Consistent Extractions... … BN.com

16. Demo!

17. Observation: Complementary Extraction – Most sources have some fields extracted correctly – Most fields could be extracted correctly in some sources o:correct x:incorrect (c) TreeAlign (WWW’05) (a) RoadRunner (VLDB’01) (b) ExAlg (SIGMOD’03) titleauthors amazon bn.com

18. Our Solution: Synchronization Synchronize a source with its context Domain Knowledge ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: New sources: Amazon buy.com borders powells AnotherBook.com The Context of AnotherBook.com … … … … Consistent Extractions... … BN.com Context-Aware Wrapping

19. Our Solution: Synchronization Synchronize a source with its context … … … …... Consistent Extractions Wrapper A …… Wrapper C Auto-Wrap …… Auto-Wrap Context-Aware Wrapping Domain Knowledge ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: New sources: Amazon buy.com borders powells AnotherBook.com The Context of AnotherBook.com … … BN.com Initial Extractions

20. Our Solution: Synchronization Synchronize a source with its context … … … …... Consistent Extractions Wrapper A Initial Extractions …… Wrapper C Auto-Wrap …… Auto-Wrap Synchronization Context-Aware Wrapping Domain Knowledge ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: New sources: Amazon buy.com borders powells AnotherBook.com The Context of AnotherBook.com … … BN.com

21. Our Solution: Synchronization Synchronize a source with its context … … … …... Consistent Extractions Wrapper A Initial Extractions …… Wrapper C Auto-Wrap …… Auto-Wrap Synchronization Context-Aware Wrapping Domain Knowledge ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: New sources: Amazon buy.com borders powells AnotherBook.com The Context of AnotherBook.com … … BN.com But, How to Synchronize?

22. Intuition: Wrapping as Communication  Sources encode records as Web pages  Wrappers decode Web pages back to records What is the best possible decoding scheme? SourceWrapper (Encoder) (Decoder) records HTML pages (Code)

23. Encoding & Decoding a Single Message EncoderDecoder Channel 100 100-10 Message SourceWrapper records HTML pages The Art of....…Hardcover$143.65 The Art of....…Hardcover $143.65 The Art of …... Hardcover $143.65 101-10 10X error

24. Insight: Turbo Codes – Multiple Messages EncoderDecoder Channel Encoder 2Decoder 2 Message Reinforcing Source 2Wrapper 2 C Prog ……Paperback…$44.20 C Prog… /a> Paperback Mar 1988 $44.20 Source 1Wrapper 1 records HTML pages The Art of....…Hardcover$143.65The Art of....…Hardcover $143.65 C……Paperback Mar 1988$44.20 100 100-10101-10 100-11100-01 X00 10X 100 The Art of …... Hardcover $143.65 error

25. Context-Aware v.s. Single-Source Wrapping Traditional single-source wrapping: Utilize the information within a source (syntax) Context-aware wrapping: Further utilize the information between multiple sources (content) records HTML pages The Art of....…Hardcover$143.65 The Art of....…Hardcover $143.65 C……Paperback Mar 1988$44.20 The Art of …... Hardcover $143.65 C Prog… /a> Paperback Mar 1988 $44.20 Source 2Wrapper 2 Source 1Wrapper 1 C Prog ……Paperback…$44.20

26. Challenges to Resolve Source 2Wrapper 2 Source 1Wrapper 1 records HTML pages The Art of....…Hardcover$143.65 The Art of....…Hardcover $143.65 C……Paperback Mar 1988$44.20 The Art of …... Hardcover $143.65 C Prog… /a> Paperback Mar 1988 $44.20 1.Is there really a common message? 2.How to reinforce between sources for “corresponding” pieces? C Prog ……Paperback…$44.20

27. Challenge 1: Is there a common message? From our observation  The fields converge to a small set  domain fields Hidden domain model:  ={ f 1, f 2, … }  Each f i is a statistic model for generating a field  Example: Book domain title: 2-20 randomly picked-up words authors: a few randomly picked-up non-dictionary words format: 1-2 words from {paperback, hardcover, …} date: + or mm/dd/yyyy price: $............

28. Message: Hidden Domain Model From our observation  Each source is a subset of domain fields Abstraction of encoding records as pages Message

29. Challenge 2: How to reinforce between sources? Determine Field Correspondence  Schema Matching S4:S4: f 11 f 12 f 21 f 22 f 23 f 14 f 13 f 31 f 32 f 34 f 45 f 44 f 42 f 41 S3:S3: S2:S2: S1:S1: 22 33 44 11 The Da Vinci Code Lord of the Rigns Hanegraaff & Maier Tolkien …… UK Edition 1st Edition 5.29 EUR 6.38 EUR C Programming Lang. Effective Java PL guide B. Kernighan et al J. Bloch Mar 1988 Paperback Jun 2001 ………… $44.20 $44.99 C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) Britchie. Kernighan Biarne Stroustrup ………… Price: $39.48 Price: $51.99 The Art of Comp. Prog. Modern Operating Systemsh D. Knuth A. Tanenbaum … Hardcover $143.65 Paperback $45.99

30. Reinforcing: Soft Schema Matching If two sources are consistent, their matching score will be high; otherwise, low. 22 33 44 S4:S4: f 11 f 12 f 21 f 22 f 23 f 14 f 13 f 31 f 32 f 34 f 45 f 44 f 42 f 41 S3:S3: S2:S2: S1:S1: The Da Vinci Code Lord of the Rigns Hanegraaff & Maier Tolkien …… UK Edition 1st Edition 5.29 EUR 6.38 EUR C Programming Lang. Effective Java PL guide B. Kernighan et al J. Bloch Mar 1988 Paperback Jun 2001 ………… $44.20 $44.99 C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) Britchie. Kernighan Biarne Stroustrup ………… Price: $39.48 Price: $51.99 The Art of Comp. Prog. Modern Operating Systemsh D. Knuth A. Tanenbaum … Hardcover $143.65 Paperback $45.99

31. Reinforcing: Soft Schema Matching If two sources are consistent, their matching score will be high; otherwise, low.  1* 1* Y2*Y2* Y3*Y3* Y4*Y4* 2*2* 3*3* 4*4* S4:S4: f 11 f 12 f 21 f 22 f 23 f 14 f 13 f 31 f 32 f 34 f 45 f 44 f 42 f 41 S3:S3: S2:S2: S1:S1: The Da Vinci Code Lord of the Rigns Hanegraaff & Maier Tolkien …… UK Edition 1st Edition 5.29 EUR 6.38 EUR C Programming Lang. Effective Java PL guide B. Kernighan et al J. Bloch Mar 1988 Paperback Jun 2001 ………… $44.20 $44.99 C Prog. Lang. (2nd Edition) C++ Prog. Lang. (3nd Edition) Britchie. Kernighan Biarne Stroustrup ………… Price: $39.48 Price: $51.99 The Art of Comp. Prog. Modern Operating Systemsh D. Knuth A. Tanenbaum … Hardcover $143.65 Paperback $45.99

32. Implementation A field is modeled as a graph model: Each node (L or V) is a prob. distribution over Examples: $35.07 Our Price: $34.07 Low Price: $23.05 ISBN: 012569586161 UPC# 014633147841 L abel V alue

33. Experiment Setup Domains (each, 10 sources)  Albums, e.g., allmusic.com, etc  Books, e.g., amazon.com, bn.com, etc  Cars, e.g., cars.com  The 1st response pages for 3 queries (~300 records in a domain) Base automatic single-source wrappers  LL: Line-Splitter (baseline)  RR: RoadRunner (VLDB’01)  EA: ExAlg (SIGMOD’03)  TA: TreeAlign (WWW’05)

34. Experiment Result: 10 New Book Sources  Achieve lower inconsistency (i.e., lower error rate)  Achieve stabler performance (regardless the initial single-source wrappers) Error rate: LL:.33->.05 RR:.38->.06 EA:.15->.05 TA:.35->.06

35. Experiment Result: Incremental Setting  More sources do achieve better performance Base Wrapper: LL

36. Conclusions We proposed and developed  Concept: Context-aware wrapping to support data wrapping in integrating online sources  Approach: Synchronized data extraction from the insight of turbo codes Or, in a nutshell:  Evaluation: Experiments show the feasibility of the proposed approach Wrapping (Data Extraction) Schema Matching

37. The End Thanks!!