Presentation is loading. Please wait.

Presentation is loading. Please wait.

1/1/ Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "1/1/ Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh."— Presentation transcript:

1 1/1/ Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh Day 1, 2, Chorng-Shyong Ong 2, and Wen-Lian Hsu 1,*, Fellow, IEEE 1 Institute of Information Science, Academia Sinica, Taiwan 2 Department of Information Management, National Taiwan University, Taiwan {myday, hsu}@iis.sinica.edu.tw; ongcs@im.ntu.edu.tw IEEE IRI 2007, Las Vegas, Nevada, USA, Aug 13-15, 2007.

2 Min-Yuh Day (NTU; SINICA) 2/2/ Outline Introduction Research Background Methods Hybrid GA-CRF-SVM Architecture Experimental Design Experimental Results and Discussion Conclusions

3 Min-Yuh Day (NTU; SINICA) 3/3/ Introduction Question classification (QC) plays an important role in cross-language question answering (CLQA) QC: Accurately classify a question in to a question type and then map it to an expected answer type “ What is the biggest city in the United States? ” Question Type: “ Q_LOCATION_CITY ” Extract and filter answers in order to improve the overall accuracy of a cross-language question answering system

4 Min-Yuh Day (NTU; SINICA) 4/4/ Introduction (cont.) Question informers (QI) play a key role in enhancing question classification for factual question answering QI: Choosing a minimal, appropriate contiguous span of a question token, or tokens, as the informer span of a question, which is adequate for question classification. “ What is the biggest city in the United States? ” Question informer: “ city ” “ city ” is the most important clue in the question for question classification.

5 Min-Yuh Day (NTU; SINICA) 5/5/ Introduction (cont.) Feature Selection in Machine Learning Optimization problem that involves choosing an appropriate feature subset. Hybrid approach that integrates Genetic Algorithm (GA) and Conditional Random Fields (CRF) improves the accuracy of question informer prediction in traditional CRF models (Day et al., 2006) We propose an integrated Genetic Algorithm (GA) and Machine Learning (ML) approach for question classification in cross-language question answering.

6 Min-Yuh Day (NTU; SINICA) 6/6/ Research Background Cross Language Question Answering International Question Answering (QA) contests TREC QA: 1999~ Monolingual QA in English QA@CLEF: 2003~ European languages in both non-English monolingual and cross-language NTCIR CLQA: 2005~ Asian languages in both monolingual and cross-language Question Classification Rule-based method Machine Learning based method

7 Min-Yuh Day (NTU; SINICA) 7/7/ Research Background (cont.) Two strategies for question classification in English-Chinese cross-language question answering 1) Chinese QC (CQC) for both English and Chinese queries. English source language has to be translated into the Chinese target language in advance. 2) English QC (EQC) for English queries and Chinese QC (CQC) for Chinese queries. We focus on question classification in English- Chinese cross-language question answering Bilingual QA system for English source language queries and Chinese target document collections.

8 Min-Yuh Day (NTU; SINICA) 8/8/ Methods Hybrid GA-CRF-SVM Architecture GA for CRF Feature Selection GA-CRF Question Informer Prediction SVM-based Question Classification using GA- CRF Question Informer

9 Hybrid GA-CRF-SVM Architecture CRF-based Question Informer Prediction GA for CRF Feature Selection Near Optimal Feature Subset of CRF Near Optimal CRF Prediction Model Question GA-CRF Question Informer Prediction Question Informer SVM-based Question Classification Question Type SVM-based Question Classification GA : Feature Selection GA CRF SVM

10 Encoding a Feature Subset of CRF with the structure of chromosomes Initialization Evaluate (Fitness Function) Population CRF model 10-fold Cross Validation GA Operators: Reproduction, Crossover, Mutation x: Feature subset F(x):Fitness Function Stopping criteria Satisfied? Near Optimal Feature Subset of CRF Near Optimal CRF Prediction Model Training dataset Test dataset CRF-based Question Informer Prediction GA-CRF Learning Yes No GA-CRF Question Informer Prediction

11 Min-Yuh Day (NTU; SINICA) 11/ Experiment Design Data set for English Question Classification Training dataset (5288E) 4,204 questions from UIUC QC dataset (E) + 500 questions from the NTCIR-5 CLQA development set (E) + 200 questions from the NTCIR-5 CLQA test set (E) + 384 questions from TREC2002 questions (E) Test dataset (CLQA2T150E) 150 English questions from NTCIR-6 CLQA ’ s formal run Data set for Chinese Question Classification Training dataset (2322C) 1238 question from IASL (C) + 500 questions from the NTCIR-5 CLQA development set (C) + 200 questions from the NTCIR-5 CLQA test set (C) + 384 questions from TREC2002 questions (translated) (C) Test dataset (CLQA2T150C) 150 Chinese questions from NTCIR-6 CLQA ’ s formal run

12 Min-Yuh Day (NTU; SINICA) 12/ Experiment Design (cont.) Features for English Question Classification Syntactic features Word-based bi-grams of the question (WB) First word of the question (F1) First two words of the question (F2) Wh-word of the question (WH) i.e., 6W1H1O: who, what, when, where, which, why, how, and other Semantic features Question informers predicted by the GA-CRF model (QIF) Question informer bi-grams predicted by the GA- CRF model (QIFB)

13 Min-Yuh Day (NTU; SINICA) 13/ Experiment Design (cont.) Features for Chinese Question Classification Syntactic features Bag-of-Words character-based bi-grams (CB) word-based bi-grams (WB). Part-of-Speech (POS) Semantic Features HowNet Senses HowNet Main Definition (HNMD) HowNet Definition (HND). TongYiCi CiLin (TYC)

14 Min-Yuh Day (NTU; SINICA) 14/ Experiment Design (cont.) Performance Metrics Accuracy MRR (mean reciprocal rank) where rank i is the rank of the first corrected question type of the i th question, and M is total number of questions.

15 Min-Yuh Day (NTU; SINICA) 15/ Experimental Results Question informer prediction Using GA to optimize the selection of the feature subset in CRF-based question informer prediction improves the F-score from 88.9% to 93.87%, and reduces the number of features from 105 to 40. Training dataset (UIUC Q5500) Test dataset (UIUC Q500) The accuracy of our proposed GA-CRF model for the UIUC dataset is 95.58% compared to 87% for the traditional CRF model reported by Krishnan et al.(2005) The proposed hybrid GA-CRF model for question informer prediction significantly outperforms the traditional CRF model.

16 Min-Yuh Day (NTU; SINICA) 16/ Experimental Results English Question Classification (EQC) using SVM

17 Min-Yuh Day (NTU; SINICA) 17/ Experimental Results of Chinese Question Classification (CQC) using SVM with different features Feature Used Top 1 Accuracy (Fine) Top 1 Accuracy (Coarse) Top 5 MRR (Fine) Top 5 MRR (Coarse) POS53.33%65.33%0.57320.7533 POSB60.00%74.00%0.64690.7970 HNMD71.33%81.33%0.74800.8832 CB74.00%84.67%0.79340.9130 HNMDB74.00%86.00%0.79160.9117 C74.67%84.67%0.79790.9152 TYCB74.67%86.00%0.78800.9062 HND74.67%86.67%0.78600.9102 W76.00%88.00%0.79010.9208 HNDB76.67%88.00%0.80000.9162 WB77.33%88.00%0.80670.9162 TYC77.33%88.67%0.80190.9240

18 Min-Yuh Day (NTU; SINICA) 18/ Chinese Question Classification (CQC) using SVM Experimental Results (cont.)

19 Min-Yuh Day (NTU; SINICA) 19/ Conclusions We have proposed a hybrid genetic algorithm and machine learning approach for cross-language question classification. The major contribution of this paper is that the proposed approach enhances cross-language question classification by using the GA-CRF question informer feature with Support Vector Machines (SVM). The results of experiments on NTCIR-6 CLQA question sets demonstrate the efficacy of the approach in improving the accuracy of question classification in English-Chinese cross-language question answering.

20 Min-Yuh Day (NTU; SINICA) 20/ Applications: ASQA (Academia Sinica Question Answering System) ASQA (IASL-IIS-SINICA-TAIWAN) ASQA is the best performing Chinese question answering system. The first place in the English-Chinese (E-C) subtask of the NTCIR-6 Cross-Lingual Question Answering (CLQA) task.(2007) The first place in the Chinese-Chinese (C-C) subtask of the NTCIR-6 Cross-Lingual Question Answering (CLQA) task.(2007) The first place in the Chinese-Chinese (C-C) subtask of the NTCIR-5 Cross-Lingual Question Answering (CLQA) task.(2005) http://asqa.iis.sinica.edu.tw

21 Min-Yuh Day (NTU; SINICA) 21/ Q & A Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh Day 1, 2, Chorng-Shyong Ong 2, and Wen-Lian Hsu 1,*, Fellow, IEEE 1 Institute of Information Science, Academia Sinica, Taiwan 2 Department of Information Management, National Taiwan University, Taiwan {myday, hsu}@iis.sinica.edu.tw; ongcs@im.ntu.edu.tw IEEE IRI 2007, Las Vegas, Nevada, USA, Aug 13-15, 2007.

Download ppt "1/1/ Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh."

Similar presentations

Recognizing Human Actions by Attributes CVPR2011 Jingen Liu, Benjamin Kuipers, Silvio Savarese Dept. of Electrical Engineering and Computer Science University.

Recognizing Human Actions by Attributes CVPR2011 Jingen Liu, Benjamin Kuipers, Silvio Savarese Dept. of Electrical Engineering and Computer Science University.
Particle swarm optimization for parameter determination and feature selection of support vector machines Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen,

Particle swarm optimization for parameter determination and feature selection of support vector machines Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen,
1/1/ A Knowledge-based Approach to Citation Extraction Min-Yuh Day 1,2, Tzong-Han Tsai 1,3, Cheng-Lung Sung 1, Cheng-Wei Lee 1, Shih-Hung Wu 4, Chorng-Shyong.

1/1/ A Knowledge-based Approach to Citation Extraction Min-Yuh Day 1,2, Tzong-Han Tsai 1,3, Cheng-Lung Sung 1, Cheng-Wei Lee 1, Shih-Hung Wu 4, Chorng-Shyong.
A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts 04 10, 2014 Hyun Geun Soo Bo Pang and Lillian Lee (2004)

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts 04 10, 2014 Hyun Geun Soo Bo Pang and Lillian Lee (2004)
Recognition of Fragmented Characters Using Multiple Feature-Subset Classifiers Recognition of Fragmented Characters Using Multiple Feature-Subset Classifiers.

Recognition of Fragmented Characters Using Multiple Feature-Subset Classifiers Recognition of Fragmented Characters Using Multiple Feature-Subset Classifiers.
1/1/ Integrating Genetic Algorithms with Conditional Random Fields to Enhance Question Informer Prediction Min-Yuh Day 1, 2, Chun-Hung Lu 1, 2, Chorng-Shyong.

1/1/ Integrating Genetic Algorithms with Conditional Random Fields to Enhance Question Informer Prediction Min-Yuh Day 1, 2, Chun-Hung Lu 1, 2, Chorng-Shyong.
Daozheng Chen 1, Mustafa Bilgic 2, Lise Getoor 1, David Jacobs 1, Lilyana Mihalkova 1, Tom Yeh 1 1 Department of Computer Science, University of Maryland,

Daozheng Chen 1, Mustafa Bilgic 2, Lise Getoor 1, David Jacobs 1, Lilyana Mihalkova 1, Tom Yeh 1 1 Department of Computer Science, University of Maryland,
1/1/ An Integrated Knowledge-based and Machine Learning Approach for Chinese Question Classification Min-Yuh Day 1,2, Cheng-Wei Lee 1, Shih-Hung Wu 3,

1/1/ An Integrated Knowledge-based and Machine Learning Approach for Chinese Question Classification Min-Yuh Day 1,2, Cheng-Wei Lee 1, Shih-Hung Wu 3,
1 Integrating User Feedback Log into Relevance Feedback by Coupled SVM for Content-Based Image Retrieval 9-April, 2005 Steven C. H. Hoi *, Michael R. Lyu.

1 Integrating User Feedback Log into Relevance Feedback by Coupled SVM for Content-Based Image Retrieval 9-April, 2005 Steven C. H. Hoi *, Michael R. Lyu.
Introduction to CL Session 1: 7/08/2011. What is computational linguistics? Processing natural language text by computers  for practical applications.

Introduction to CL Session 1: 7/08/2011. What is computational linguistics? Processing natural language text by computers  for practical applications.
Automatic Set Expansion for List Question Answering Richard C. Wang, Nico Schlaefer, William W. Cohen, and Eric Nyberg Language Technologies Institute.

Automatic Set Expansion for List Question Answering Richard C. Wang, Nico Schlaefer, William W. Cohen, and Eric Nyberg Language Technologies Institute.
1/1/ Designing an Ontology-based Intelligent Tutoring Agent with Instant Messaging Min-Yuh Day 1,2, Chun-Hung Lu 1,3, Jin-Tan David Yang 4, Guey-Fa Chiou.

1/1/ Designing an Ontology-based Intelligent Tutoring Agent with Instant Messaging Min-Yuh Day 1,2, Chun-Hung Lu 1,3, Jin-Tan David Yang 4, Guey-Fa Chiou.
Active Learning Strategies for Compound Screening Megon Walker 1 and Simon Kasif 1,2 1 Bioinformatics Program, Boston University 2 Department of Biomedical.

Active Learning Strategies for Compound Screening Megon Walker 1 and Simon Kasif 1,2 1 Bioinformatics Program, Boston University 2 Department of Biomedical.
Genetic Algorithm What is a genetic algorithm? “Genetic Algorithms are defined as global optimization procedures that use an analogy of genetic evolution.

Genetic Algorithm What is a genetic algorithm? “Genetic Algorithms are defined as global optimization procedures that use an analogy of genetic evolution.
Text Classification Using Stochastic Keyword Generation Cong Li, Ji-Rong Wen and Hang Li Microsoft Research Asia August 22nd, 2003.

Text Classification Using Stochastic Keyword Generation Cong Li, Ji-Rong Wen and Hang Li Microsoft Research Asia August 22nd, 2003.
Online Stacked Graphical Learning Zhenzhen Kou +, Vitor R. Carvalho *, and William W. Cohen + Machine Learning Department + / Language Technologies Institute.

Online Stacked Graphical Learning Zhenzhen Kou +, Vitor R. Carvalho *, and William W. Cohen + Machine Learning Department + / Language Technologies Institute.
Attention Deficit Hyperactivity Disorder (ADHD) Student Classification Using Genetic Algorithm and Artificial Neural Network S. Yenaeng 1, S. Saelee 2.

Attention Deficit Hyperactivity Disorder (ADHD) Student Classification Using Genetic Algorithm and Artificial Neural Network S. Yenaeng 1, S. Saelee 2.
Slide Image Retrieval: A Preliminary Study Guo Min Liew and Min-Yen Kan National University of Singapore Web IR / NLP Group (WING)

Slide Image Retrieval: A Preliminary Study Guo Min Liew and Min-Yen Kan National University of Singapore Web IR / NLP Group (WING)
Evaluating the Contribution of EuroWordNet and Word Sense Disambiguation to Cross-Language Information Retrieval Paul Clough 1 and Mark Stevenson 2 Department.

Evaluating the Contribution of EuroWordNet and Word Sense Disambiguation to Cross-Language Information Retrieval Paul Clough 1 and Mark Stevenson 2 Department.
Kuang Ru; Jinan Xu; Yujie Zhang; Peihao Wu Beijing Jiaotong University

Kuang Ru; Jinan Xu; Yujie Zhang; Peihao Wu Beijing Jiaotong University

Similar presentations

Ads by Google