1. Cost-aware Pre-training for Multiclass Cost-sensitive Deep Learning
Source : Computer Science, 2015.
Authors : Yuan C, Lin H T, Yang S W
Speaker : Jiefan Tan
Date : 2017/5/25
國立台灣大學資訊工程學系

2. Outline Introduction Pre-training with AutoEncoder
Cost Matrix,
Cost-sensitive methods
Pre-training with AutoEncoder
Cost-sensitive AutoEncoder
Experimental results
Conclusion

3. Cost Matrix For each sample（x, y, C） C(y, k) k∈[1, K] Donate
Predict
Actual
Donate
Not to donate
（Mailing ）
Get donation（-6+x） -6
（No mailing ）
-x+6
For each sample（x, y, C）
C(y, k) k∈[1, K]

4. Cost-Sensitive Methods
Bayes-optimal decision(BOD)
Trains a classifier that outputs class-wise probability
Predicts with 𝑦 = argmin 1≤𝑗≤𝐾 𝑘=1 𝐾 𝑝 𝑦=𝑘 𝑥 𝐶(𝑘,𝑗)
Regression-based:
Train a set of regressors rk , where k∈{1,2,…K}
such that rk(x) ≈ C(y, k)
Predicts with 𝑦 = argmin 1≤𝑗≤𝐾 𝑟𝑘(𝑥)

5. AutoEncoder layer-wise pre-training
To make the network parameters start from a good initial point

6. AutoEncoder y = S(Wx + b) x’ = S(WTy + b’)
均方误差
交叉熵
AE reconstructs the original input x
Goal of AE:

7. Cost-sensitive AutoEncoder
y = S(Wx + b)
x’ = S(WTy + b’)
x’’ = W’y+b’’
Goal of CSAE:
1. Reconstruct the original input x ;
2. Digest the cost information by reconstructing the cost vector C.

8. AutoEncoder Fine-tuning

9. Experiment & Results
Dataset：
MNIST
bg-img-rot
SVHN
CIFAR-10

10. Cost Setting Cost matrix K*K C(y,y) = 0 C(k,y)
eg： class A = 10 ; class B = 50
C(A:B) = 10*(60:10)/(60:50) = 50
C(B:A) = 10*(60:50)/(60:10) =2
将数据集的类分布加入代价定义，少数类的误分代价将更高

11. Experiment & Results
Performance with β

12. Experiment & Results

13. Conclusion Leaning a method to define cost C
For CNN, may replace softmax layer with cost- ware loss layer

14. Thank you!