Presentation is loading. Please wait.

Presentation is loading. Please wait.

Evidence of Quality of Textual Features on the Web 2.0 Flavio Figueiredo David FernandesEdleno MouraMarco Cristo Fabiano BelémHenrique.

Published byClemence Holmes Modified over 9 years ago

Similar presentations

Presentation on theme: "Evidence of Quality of Textual Features on the Web 2.0 Flavio Figueiredo David FernandesEdleno MouraMarco Cristo Fabiano BelémHenrique."— Presentation transcript:

1 Evidence of Quality of Textual Features on the Web 2.0 Flavio Figueiredo flaviov@dcc.ufmg.br David FernandesEdleno MouraMarco Cristo Fabiano BelémHenrique PintoJussara AlmeiraMarcos Gonçalves UFMG UFAMFUCAPI BRAZIL

2 Motivation Web 2.0 Huge amounts of multimedia content Information Retrieval Mainly focused on text (i.e. Tags) User generated content No guarantee of quality How good are these textual features for IR?

3 User Generated Content

4

5

6 Textual Features

7 Multimedia Object

8 Textual Features Multimedia Object TITLE

9 Textual Features Multimedia Object TITLE DESCRIPTION

10 Textual Features Multimedia Object TITLE DESCRIPTION TAGS

11 Textual Features Multimedia Object TITLE DESCRIPTION TAGS COMMENTS

12 Textual Features Textual Features TITLE DESCRIPTION TAGS COMMENTS

13 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity

14 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity Analyze the quality of features for object classification

15 Applications/Features Applications Textual Features Title – Tags – Descriptions – Comments

16 Data Collection June / September / October 2008 CiteULike - 678,614 Scientific Articles LastFM - 193,457 Artists Yahoo Video! - 227,252 Objects YouTube - 211,081 Objects Object Classes Yahoo Video! And YouTube - Readily Available LastFM - AllMusic Website (~5K artists)

17 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity

18 Textual Feature Usage Percentage of objects with empty features (zero terms) TITLETAGDESC.COMM. CiteULike0.53%8.26%51.08% 99.96% LastFM0.00% 18.88% 53.52%53.38% YahooVid.0.15% 16.00% 1.17% 96.88% Youtube0.00%0.06%0.00%23.36% Restrictive features more present Tags can be absent in 16% of content RestrictiveCollaborative

19 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity

20 Amount of Content Vocabulary size (average number of unique stemmed terms) per feature TITLETAGDESC.COMM. CiteULike7.54.065.251.9 LastFM1.827.490.1110.2 YahooVid.6.312.821.652.2 Youtube4.610.040.4322.3 TITLE < TAG < DESC < COMMENT RestrictiveCollaborative

21 Amount of Content Vocabulary size (average number of unique stemmed terms) per feature TITLETAGDESC.COMM. CiteULike7.54.065.251.9 LastFM1.827.490.1110.2 YahooVid.6.312.821.652.2 Youtube4.610.040.4322.3 Collaboration can increase vocabulary size RestrictiveCollaborative

22 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity

23 Descriptive Capacity Term Spread (TS) TS(DOLLS) =2

24 Descriptive Capacity Term Spread (TS) TS(DOLLS) =2 TS(PUSSYCAT) =2

25 Descriptive Capacity Feature Instance Spread (FIS) TS(DOLLS) =2 TS(PUSSYCAT) =2 FIS(TITLE) = (TS(DOLLS) + TS(PUSSYCAT)) / 2 = 4/2 = 2

26 Descriptive Capacity Average Feature Spread (AFS) – Given by the average FIS across the collection TITLE TAGDESC.COMM. CiteULike 1.91 1.621.12- LastFM 2.65 1.321.211.20 YahooVid. 2.26 1.861.51- Youtube 2.53 2.071.721.12 TITLE > TAG > DESC > COMMENT

27 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity

28 Discriminative Capacity Inverse Feature Frequency (IFF) Based on Inverse Document Frequency (IDF)

29 Bad Discriminator “video” Discriminative Capacity Inverse Feature Frequency (IFF) Youtube

30 Bad Discriminator “video” Good. “music” Discriminative Capacity Inverse Feature Frequency (IFF) Youtube

31 Bad Discriminator “video” Good. “music” Great. “CIKM” Noise. “v1d30” Discriminative Capacity Inverse Feature Frequency (IFF) Youtube

32 Average Inverse Feature Frequency (AIFF) – Average of IFF across the collection TITLETAGDESC.COMM. CiteULike7.31 7.59 7.02- LastFM 6.64 6.005.835.90 YahooVid. 6.67 6.546.37- Youtube 7.12 7.007.736.64 (TITLE or TAG) > DESC > COMMENT Discriminative Capacity

33 Research Goals Characterize evidence of quality of textual features Usage Amount of content Descriptive capacity Discriminative capacity Analyze the quality of features for object classification

34 Object Classes

35 Vector Space Features as vectors

36 Vector Combination Average fraction of common terms (Jaccard) between top FIVE TSxIFF terms of features CiteULLastFMYahooV.Youtube TITLE X TAGS0.130.07 0.520.36 TITLE X DESC 0.31 0.22 0.400.28 TAGS X DESC0.13 0.430.32 TITLE X COMM - 0.12-0.14 TAGS X COMM-0.10-0.17 DESC X COMM-0.18-0.16 Bellow 0.52. Significant amount of new content

37 Vector Combination Feature combination using concatenation Title: Tags: Bag-of-Words: Title: Tags: Result:

38 Vector Combination Feature combination using Bag-of-word Title: Tags: Result:

39 Term Weight Term weight TSTFIFF TS x IFF TF x IFF

40 Object Classification Support vector machines Vectors TITLE, TAG, DESCRIPTION or COMMENT CONCATENATION BAG OF WORDS Term weight TSTFIFF TS x IFF TF x IFF

41 Classification Results LastFMYahooV.Youtube TITLE0.200.520.40 TAG0.800.630.54 DESCRIPTION0.750.570.43 COMMENT0.52-0.46 CONCAT0.800.660.59 BAGOW0.800.660.56 Macro F1 results for TSxIFF Bad results inspite good descripive/discriminative capacity Impact due to the small amount of content

42 Classification Results LastFMYahooV.Youtube TITLE0.200.520.40 TAG0.800.630.54 DESCRIPTION0.750.570.43 COMMENT0.52-0.46 CONCAT0.800.660.59 BAGOW0.800.660.56 Macro F1 results for TSxIFF Best Results Good descriptive/discriminative capacity Enough content

43 Classification Results LastFMYahooV.Youtube TITLE0.200.520.40 TAG0.800.630.54 DESCRIPTION0.750.570.43 COMMENT0.52-0.46 CONCAT0.800.660.59 BAGOW0.800.660.56 Macro F1 results for TSxIFF Combination brings improvement Similar insights for other weights

44 Conclusions Characterization of Quality Collaborative features more absent Different amount of content per feature Smaller features are best descriptors and discriminators New content in each feature Classification Experiment TAGS are the best feature in isolation Feature combination improves results

Download ppt "Evidence of Quality of Textual Features on the Web 2.0 Flavio Figueiredo David FernandesEdleno MouraMarco Cristo Fabiano BelémHenrique."

Similar presentations

Text Categorization.

Text Categorization.
Chapter 5: Introduction to Information Retrieval

Chapter 5: Introduction to Information Retrieval
Multimedia Database Systems

Multimedia Database Systems
Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)

Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)
LYRIC-BASED ARTIST NETWORK METHODOLOGY Derek Gossi CS 765 Fall 2014.

LYRIC-BASED ARTIST NETWORK METHODOLOGY Derek Gossi CS 765 Fall 2014.
GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.

GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.
Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)

Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)
Learning for Text Categorization

Learning for Text Categorization
Tagging Systems Austin Wester. Tags A keywords linked to a resource (image, video, web page, blog, etc) by users without using a controlled vocabulary.

Tagging Systems Austin Wester. Tags A keywords linked to a resource (image, video, web page, blog, etc) by users without using a controlled vocabulary.
Information Retrieval Ling573 NLP Systems and Applications April 26, 2011.

Information Retrieval Ling573 NLP Systems and Applications April 26, 2011.
Tagging Systems Mustafa Kilavuz. Tags A tag is a keyword added to an internet resource (web page, image, video) by users without relying on a controlled.

Tagging Systems Mustafa Kilavuz. Tags A tag is a keyword added to an internet resource (web page, image, video) by users without relying on a controlled.
Information Retrieval Review

Information Retrieval Review
Query Operations: Automatic Local Analysis. Introduction Difficulty of formulating user queries –Insufficient knowledge of the collection –Insufficient.

Query Operations: Automatic Local Analysis. Introduction Difficulty of formulating user queries –Insufficient knowledge of the collection –Insufficient.
Ch 4: Information Retrieval and Text Mining

Ch 4: Information Retrieval and Text Mining
Modern Information Retrieval Chapter 2 Modeling. Can keywords be used to represent a document or a query? keywords as query and matching as query processing.

Modern Information Retrieval Chapter 2 Modeling. Can keywords be used to represent a document or a query? keywords as query and matching as query processing.
Web Mining Research: A Survey

Web Mining Research: A Survey
Recommender systems Ram Akella February 23, 2011 Lecture 6b, i290 & 280I University of California at Berkeley Silicon Valley Center/SC.

Recommender systems Ram Akella February 23, 2011 Lecture 6b, i290 & 280I University of California at Berkeley Silicon Valley Center/SC.
MusicSense: Contextual Music Recommendation using Emotional Allocation Modeling Rui Cai, Chao Zhang, Chong Wang, Lei Zhang, and Wei-Ying Ma Proceedings.

MusicSense: Contextual Music Recommendation using Emotional Allocation Modeling Rui Cai, Chao Zhang, Chong Wang, Lei Zhang, and Wei-Ying Ma Proceedings.
Retrieval Models II Vector Space, Probabilistic.  Allan, Ballesteros, Croft, and/or Turtle Properties of Inner Product The inner product is unbounded.

Retrieval Models II Vector Space, Probabilistic.  Allan, Ballesteros, Croft, and/or Turtle Properties of Inner Product The inner product is unbounded.
Modern Information Retrieval Chapter 2 Modeling. Can keywords be used to represent a document or a query? keywords as query and matching as query processing.

Modern Information Retrieval Chapter 2 Modeling. Can keywords be used to represent a document or a query? keywords as query and matching as query processing.

Similar presentations

Ads by Google