Introduction of SNoW (Sparse Network of Winnows ) IRLab-LA Group hjLiu 2005-4-8

Introdution The SNoW Architecture File Formats Using SNoW Applying SNoW to my work

Introdution Multi-class classifier Learning architecture framework Including a true multi-class capability Standard one-vs-all training policy Predictions are done via a winner-take-all policy or a voted combination of several learners. Learning architecture framework A sparse network of sparse linear functions over a predefined or incrementally acquired feature space The user designs an architecture within that framework (defining many more parameters of the architecture )

The SNoW Architecture

The Basic System Iuput: features layer Output: Target nodes A two layer network is maintained Iuput: features layer Output: Target nodes Target nodes are linked via weighted edges to (active feature) input features Prediction positive

The Basic System Initial feature weight: The predicted target for example with a set of active features is:

Basic Learning Rules Winnow Predict negative——label is positive-〉promoted Predict positive——label is negative -〉demoted Other-〉unchang Sigmoid activation:

Basic Learning Rules Perceptron Predict negative——label is positive-〉promoted Predict positive——label is negative -〉demoted Other-〉unchang Sigmoid activation:

Basic Learning Rules Naïve Bayes

Extensions to the Basic Learning Rules Modify the behavior of basic update rules: Inculde:eligibility of features, options for discarding features, conditional prediction based on a prediction threshold, and others Constraint Classification Regularization Function Approximation Sequential Model Voting: The Clouds Architecture Threshold-Relative Updating

File Formats

Example Files 样例结束符号 0, 3, 1234, 123456, 12, 987, 234, 556: 1, 7(1.5), 5, 10(0.6), 13(-3.2): Label 样例结束符号 Features Features Label strength

Example Files(cont.) 1,10391,10149,10002,10003,10004,10460,10151,10044,10393,10143,10074,10046,10144,10145,10394,10015,10016,10146,10461,10462,10463,10458,10464,10399: 1,10391,10099,10002,10003,10004,10465,10157,10158,10393,10086,10074,10046,10159,10145,10394,10015,10016,10089,10432,10333,10433,10466,10467,10399: 1,10391,10001,10002,10003,10004,10418,10163,10044,10393,10073,10074,10046,10144,10145,10394,10015,10016,10078,10395,10019,10396,10458,10459,10399: 5,10391,10164,10002,10003,10004,10165,10166,10167,10393,10073,10074,10168,10169,10170,10394,10015,10016,10078,10468,10469,10470,10471,10472,10399: 1,10391,10001,10129,10070,10004,10369,10233,10044,10393,10073,10115,10046,10176,10177,10394,10015,10077,10234,10395,10079,10473,10474,10475,10476: 6,10391,10180,10129,10070,10004,10374,10238,10044,10393,10103,10115,10046,10176,10177,10394,10015,10077,10119,10477,10182,10478,10474,10479,10476: 2,10284,10001,10129,10003,10004,10480,10163,10044,10481,10073,10074,10046,10176,10177,10482,10483,10016,10078,10290,10019,10291,10383,10384,10294: 3,10284,10001,10002,10070,10004,10484,10227,10044,10481,10103,10115,10046,10485,10486,10482,10483,10077,10119,10290,10079,10370,10487,10488,10373:

Network Files target ID priorProbability cloudConfidence activeCount nonActiveCount algorithm learnerType parameters eg. target 2 0.4 0.473593433165 42 63 winnow 1 1.35 0.8 4 0.2 ID : learnerType : featureID : activeCount updates weight eg. 1 : 2 : 34 : 13 6 0.3645

Network Files(cont.) target 5 1 1 2 48 perceptron 0 0.05 4 0.48 5 : 0 : 4294967294 : 2 7 0.63 5 : 0 : 10002 : 2 7 0.63 5 : 0 : 10004 : 2 7 0.63 target 6 1 1 1 49 perceptron 0 0.05 4 0.48 target 7 1 1 1 49 perceptron 0 0.05 4 0.48 target 8 1 1 2 48 perceptron 0 0.05 4 0.48 8 : 0 : 4294967294 : 2 7 0.63 8 : 0 : 10002 : 2 7 0.63 8 : 0 : 10004 : 2 7 0.63

Result Files Example: -o softmax Note:Error Files See the file:resultP User defined the parameters of output mode Example: -o softmax Note:Error Files See the file:resultP

Using SNoW

Execution Modes Trainnig Testing Interactive Evaluation Server mode

Traning Mode Architecture Definition Parameters Training Parameters Command line usage snow -train -I inputfile -F networkfile [ -AaBbcdEefGgiLlMmOoPpRrSsTtuvWwz ] Architecture Definition Parameters Learning algorithms:-P,-W,-B Extention rules:-G,-O,-S,-t Training Parameters -e,-r,-s,-u and so on

Traning Mode(cont.) snow -train -I train_numeric -F snow_netP -P 0.05:1-10 -S 2 -r 3 Trainning file Network file Learing rule Other para

Testing Mode Testing Parameters -i,-w,-p and so on Output Parameters Command line usage snow -test -I inputfile -F networkfile [ -abEefGgiLlmOopRSstvwz ] Testing Parameters -i,-w,-p and so on Output Parameters -o <accuracy | winners | softmax | allpredictions | allactivations | allboth> and so on

Testing Mode(cont.) snow -test -I test_numeric -F snow_netP -w 0.1 -S 2 -o softmax >resultP Testing file Network file Save in file Other para Output mode

Applying SNoW to my work

完成过程 特征数值化 程序：train_Pro_toNum.pl 和test_Pro_toNum.pl 作用：对训练和测试样例数值化，并且生成类别和特征文件 训练样例 约170多万个样例，50多个类别，170多万个特征 训练时间：约20多分钟 测试样例 约6万个样例， Label全标成NULL 测试时间：约5分钟 输出结果格式转化 程序：transFormat.pl 实现方式： 一步分类 两步分类： （1）先对NULL 和 NON-NULL进行二元分类 （2）再对NON-NULL进行多类分类

参考文献： Snow-Uerguide.pdf（使用手册）

Thanks!