1. Motivation & Introduction Efficiency
A Saak Transform Approach to Efficient, Scalable and Robust Handwritten Digits Recognition
Yueru Chen, MCLab
Motivation & Introduction
Efficiency
Lossy Saak transform:
Replace the KLT with PCA for transform kernels computation
Weaknesses of CNNs:
efficiency, scalability and robustness
Saak transform:
a mapping from a real-valued function defined on a 3D cuboid to a 1D rectified spectral vector
# Kernels 32 64
128
256
All kernels
98.19
98.58
98.53
98.14
(4,11,16,20,17)
98.24
98.54
98.33
97.84
(4,5,8,7,9)
98.30
98.26
97.68
(4,5,5,6,7)
98.28
98.52
98.21
97.70
(4,4,4,5,5)
98.22
98.42
98.08
97.58
Classification Accuracy on MNIST dataset
Robustness
Test on Noisy MNIST dataset
Image synthesis
with multi-stage inverse Saak transforms
Method
S&P 1
S&P
Speckle
Gaussian
Random_bg
Texture_bg
LeNet-5
89.13
86.12
74.62
67.68
84.10
81.75
94.11
85.59
AlexNet
82.83
84.22
62.49
53.99
75.94
97.63
98.36
98.12
Saak
95.71
95.31
91.16
87.49
83.06
94.08
94.67
87.78
The distributions of Saak coefficients
Scalability
Size
Stage 1
Stage 2
Stage 3
Stage 4
Stage 5
50000
0.9999
0.9996
40000
0.9993
30000
0.9988
20000
0.9972
10000
0.9997
0.9992
0.9945
# Class
Stage 1
Stage 2
Stage 3
Stage 4
Stage 5 8
0.9998
0.9996
0.9942
0.9940
0.9550 6
0.9982
0.9983
0.9866
0.9639
0.5586 4
0.9993
0.9990
0.9816
0.9219
0.4557 2
0.9390
0.9672
0.6567
0.6694
0.3294
The classification results of using fewer classes in training
The cosine similarity of transform kernels
Overview of the proposed approach
Yueru Chen
Advisor: C.-C. Jay Kuo