NJUST @ CLSciSumm-2018 What to submit Task Framework Task 1A Task 1B Shutian Ma; Heng Zhang; Jin Xu; Chengzhi Zhang Department of Information Management, Nanjing University of Science and Technology, Nanjing, China, 210094 What to submit Task Framework Voting Weights of Precision, Recall and F1-Oriented 3-Classifiers System Voting System Classifiers Voting Weight Precision- oriented SVM (RBF) 0.3116 Recall- oriented 0.2192 F1- oriented 0.2565 DT 0.2617 0.5236 0.4233 LG 0.4268 0.2572 0.3202 For Task 1A, Multi-classifiers using Voting System, another 10 running results based on single classifiers. New Classifier: XGBOOST, an efficient and scalable implementation of gradient boosting framework Task 1A Feature Selection Decision tree, Logistic regression, SVM (Linear, RBF), XGBOOST Multi-Classifiers + Voting systems Average F1 when using Precision-Oriented 3-Classifiers Voting System Average F1 when using Precision-Oriented 4-Classifiers Voting System Task 1B Examples when Utilizing Rules to Expand Labeled Citation Text Manual Dictionary LLDA XGBOOST Manual dictionary + LLDA POS Dictionary + LLDA Average F1 when using Recall-Oriented 3-Classifiers Voting System Average F1 when using Recall-Oriented 4-Classifiers Voting System LLDA Assume each identified facet is a topic label and that each citation sentence is a mixture of the expert-assigned topics that can be learned. +LLDA strategy Use dictionary-labeled testing data to be testing data for prediction. Dictionary strategy Find the best order of judging facets POS dictionary POS results are VB and JJ, meet frequency threshold. Fixed Features + Selected Features Iteratively evaluate a candidate subset of selected features set Task 2 Average F1 when using F1-Oriented 3-Classifiers Voting System Average F1 when using F1-Oriented 4-Classifiers Voting System Group sentences into clusters based on its similarity with different parts of abstract. Extract sentence based on score from each cluster and combine into summary. Average F1 when #Negative/#Positive is 1 Average F1 when #Negative/#Positive is 2 Average F1 when #Negative/#Positive is 3 For Task 1B, we select the specific order according to F1. Manual Dictionary, we choose top 3. LLDA strategy, we pick the top 4. XGBOOST strategy, we also select top 4. People write summaries starting with some fixed phrases, such as “this paper”, “in this paper” or “we”. Meanwhile, the last sentence are usually about results or conclusions. Average F1 when #Negative/#Positive is 4 Average F1 when #Negative/#Positive is 5 Average F1 when #Negative/#Positive is 6 Acknowledgement 𝑆𝑐𝑜𝑟𝑒 𝑖 =2.5 𝑆 𝐽𝑎𝑐𝑐𝑎𝑟𝑑 +2.5 𝑆 𝐼𝐷𝐹 +2.5 𝑆 𝑇𝐹𝐼𝐷𝐹 +1.25 𝑆 𝐿𝑒𝑛𝑔𝑡ℎ +1.25 𝑆 𝑃𝑜𝑠𝑖𝑡𝑖𝑜𝑛 Deepest gratitude goes to my greatest teammate. Voting Weights of Precision, Recall and F1-Oriented 4-Classifiers System Voting System Classifiers Voting Weight Precision-oriented SVM (Linear) 0.2160 Recall- oriented 0.0699 F1- oriented 0.0954 SVM (RBF) 0.2443 0.2039 0.2320 DT 0.2051 0.4870 0.3829 LG 0.3346 0.2392 0.2897 Tools Contact Information Shutian Ma: mashutian0608@hotmail.com Heng Zhang: 525696532@qq.com Jin Xu: xujin@njust.edu.cn Chengzhi Zhang: zhangcz@njust.edu.cn Porter Stemmer Gensim, Word2Vec and Doc2Vec model Scikit-learn, Classifiers LDA and XGBOOST are applied with Python package