Download presentation
Presentation is loading. Please wait.
Published byJocelyn Miller Modified over 9 years ago
1
Aspect Guided Text Categorization with Unobserved Labels Dan Roth, Yuancheng Tu University of Illinois at Urbana-Champaign
2
Text Categorization An archetypical Multi-Class Classification (MCC) problem F : X → Y a document, d ∈ X, a collection of classes Y = {c 1, c 2,..., c N } Sports Health Business … Science C1 C2 C3 … C N 2
3
Motivation: what are we missing? Class labels (Y) contain information which can help classification How can we explore the label space? C1 C2 C3 … C N Sports Health Business … Science 3
4
Car Navigation Command Classification 4 XY
5
Aspect Variables 5 X Y find nearest restaurant Show me where I can eat nearby Find nearest restaurant ActionDetailModifierTopicManner z1z1 z2z2 z3z3 z4z4 z5z5 Null
6
Significance of the Aspect Variables Predicting better aspects implies predicting better class labels Adding constraints to the aspect space Predicting previously unobserved labels 6 If Topic = “restaurant”, then Action ≠ “turn” Observed Label 1. turn on the radio 2. GPS navigation Unobserved Label turn on GPS
7
Outline Car Command Text Categorization Task Data and aspects Unobserved labels Constrained Conditional Model (CCM) Aspects variables to introduce constraints Objective function Training and Inference Experimental Results Standard multiclass classification setting Predicting Unobserved Labels Conclusion 7
8
x1x1 x6x6 x2x2 x5x5 x4x4 x3x3 x7x7 X Y y1y1 Adding Constraints by Hidden Aspects Intuition: introduce structure on hidden variables
9
Adding Constraints Through Hidden Aspects x1x1 x6x6 x2x2 x5x5 x4x4 x3x3 x7x7 Y Z1 Z2 Z4 Z3 Z5 y1y1 Use constraints to capture the dependencies X
10
Objective Function of CCM 10 Weight Vector for “local” learners Aspect functions Penalty for violating the constraint. How far away is y from a “legal” assignment
11
11 Training and Inference Learning + Inference (L+I) Ignore constraints during training Inference Based Training (IBT) Consider constraints during training References to CCM (aka ILP formulation) Roth&Yih04, Has been shown useful in the context of many NLP problems: SRL, Summarization; Co-reference; Information Extraction; Transliteration 07; Punyakanok et.al 05,08; Chang et.al 07,08; Clarke&Lapata06,07; Denise&Baldrige07;Goldwasser&Roth'08; Martin,Smith&Xing'09
12
12 Learning + Inference x1x1 x6x6 x2x2 x5x5 x4x4 x3x3 x7x7 Z1 Z2 Z5 Z4 Z3 f1(x)f1(x) f2(x)f2(x) f3(x)f3(x) f4(x)f4(x) f5(x)f5(x) X Y Learning + Inference (L+I) Learn models independently Learning + Inference (L+I) Learn models independently
13
13 1111Y’Local Predictions Inference Based Training Example: Perceptron-based Global Learning x1x1 x6x6 x2x2 x5x5 x4x4 x3x3 x7x7 f1(x)f1(x) f2(x)f2(x) f3(x)f3(x) f4(x)f4(x) f5(x)f5(x) X Y 11 YTrue Global Labeling 111 Y’ Apply Constraints:
14
Outline Car Command Text Categorization Task Data and aspects Unobserved labels Constrained Conditional Model (CCM) Aspects variables to introduce constraints Objective function Training and Inference Experimental Results Standard multiclass classification setting Predicting Unobserved Labels Conclusion 14
15
Car Navigation Command Classification 15 XY
16
Evaluation Metrics Standard Accuracy The percentage of correctly labeled examples Weighted Aspect-based Metric (WAM) A weighted Hamming distance computed at the aspect level 16
17
Experiments and Evaluation Standard MCC Setting 17 AlgorithmAccuracy (%)WAM(%) Baseline67.8486.14 MCC(L+I) 71.1889.65 Error Reduction (%) 10.3925.32 AccuracyFless Kappa Human Annotation 75%0.764
18
Experiments and Evaluation Standard MCC Setting 18 AspectsCCMBaselineError Reduction(%) Topic 86.1481.5524.88 Action 88.3182.7232.35 Manner 89.9887.3520.79 Modifier 91.1589.5115.64 Detail 92.6889.5929.68
19
Experiments and Evaluation Predicting Unobserved Labels 19 AlgorithmAccuracy (%)WAM(%) Baseline0.0058.43 MCC(L+I) 28.1670.27 Error Reduction (%) 28.1628.48 Unobserved Label turn on GPS Observed Label 1. turn on the radio 2. GPS navigation
20
Conclusion Summary Text Categorization with a meaningful, structured label space A model that exploits the structure by adding hidden aspect variables Adding constraints and reformulating the task as a structure prediction problem Predicting unobserved new labels 20
21
Thank You! AND Questions? 21
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.