1. Zero shot learning Presented by: YuYing Chou
phone:
Advisor: Tyng-Luh Liu, Hsuan-Tien Lin

2. What is zero shot learning
Teach computer to recognize something they have not seen.
How to make it become possible?

3. How human start to recognize something
Imagine
Guess with hint

4. what does alien look like

5. The zero shot task is similar …
Baby tries to learn something new in the world.
我們要教machine 如何辨認斑馬

6. Give the machine hints
C. H. Lampert, H. Nickisch, and S. Harmeling. "Learning To Detect Unseen Object Classes by
Between-Class Attribute Transfer". In CVPR, 2009

7. Give the machine a hint
The attributes are the hints given to the computer.
C. H. Lampert, H. Nickisch, and S. Harmeling. "Learning To Detect Unseen Object Classes by
Between-Class Attribute Transfer". In CVPR, 2009

8. Zebra wiki
Zebras (/ˈziːbrə/ ZEE-brə, UK also /ˈzɛbrə/ ZEB-rə)[1] are several species of African equids (horse family) united by their distinctive black and white striped coats. Their stripes come in different patterns, unique to each individual. They are generally social animals that live in small harems to large herds. Unlike their closest relatives, horses and donkeys, zebras have never been truly domesticated.
There are three species of zebras: the plains zebra, the mountain zebra and the Grévy's zebra. The plains zebra and the mountain zebra belong to the subgenus Hippotigris, but Grévy's zebra is the sole species of subgenus Dolichohippus. The latter resembles an ass, to which zebras are closely related, while the former two look more horse-like. All three belong to the genus Equus, along with other living equids.
Frome, Andrea, Greg S. Corrado, Jon Shlens, Samy Bengio, Jeff Dean, and Tomas Mikolov. "Devise:
A deep visual-semantic embedding model." In Advances in neural information processing systems, pp

9. Old models

10. From

11. The drawbacks of old models
Zerba is similar to horses and has stripes
Bias
like
horse
spot

12. transductive learning
Unseen data
Unseen data
Unsupervised learning

13. Fu, Yanwei, Timothy M. Hospedales, Tao Xiang, and Shaogang Gong
Fu, Yanwei, Timothy M. Hospedales, Tao Xiang, and Shaogang Gong. "Transductive multi-view
zero-shot learning." IEEE transactions on pattern analysis and machine intelligence 37, no. 11 (2015):

14. Generative model to generate fake unseen image
decoder

15. Generate Unseen image
decoder

16. Use unseen image to train model (MLP: Multi-Level Perceptron)

17. 𝐿𝑜𝑠𝑠= 𝑖=1 𝑁 [ 𝑓 𝑖 𝑎 − 𝑓 𝑖 𝑝 2 2 − 𝑓 𝑖 𝑎 − 𝑓 𝑖 𝑛 2 2 ]
𝑓 𝑖 𝑎 : a sample of image features
( 𝑓 𝑖 𝑎 , 𝑓 𝑖 𝑝 ) : a positive pair
( 𝑓 𝑖 𝑎 , 𝑓 𝑖 𝑛 ) : a negative pair
Triplet pair
𝐿𝑜𝑠𝑠= 𝑖=1 𝑁 [ 𝑓 𝑖 𝑎 − 𝑓 𝑖 𝑝 − 𝑓 𝑖 𝑎 − 𝑓 𝑖 𝑛 ]
𝑓 𝑖 𝑎
𝑓 𝑖 𝑛
𝑓 𝑖 𝑝
𝑓 𝑖 𝑎
𝑓 𝑖 𝑛
𝑓 𝑖 𝑝
After learning

18. *method 1. CVAE (Conditional Variational Autoencoder) 2. MLP
3. Triplet loss

19. *steps 1. Generate seen classes features with CNN
2. Train CVAE with seen classes image features
3. Generate unseen classes image from CVAE
4. Train MLP with seen classes features and generated unseen classes image features

20. Caltech-UCSD Birds

21. [2]Annadani, Yashas and Biswas, Soma Preserving Semantic Relations for Zero-Shot Learning ,arXiv preprint arXiv: , 2018
[3] Chen, Long, Hanwang Zhang, Jun Xiao, Wei Liu, and Shih-Fu Chang. ”Zero-Shot Visual Recognition using
Semantics-Preserving Adversarial Embedding Network.” arXiv preprint arXiv: (2017).
[4] Arora, Gundeep, Vinay Kumar Verma, Ashish Mishra, and Piyush Rai. ”Generalized Zero-Shot Learning via Synthesized Examples.” arXiv preprint arXiv: (2017).
[6] Song, Jie, Chengchao Shen, Yezhou Yang, Yang Liu, and Mingli Song. ”Transductive Unbiased Embedding for Zero-Shot Learning.” arXiv preprint arXiv: (2018).

22. *result
The generative model very likely improves the ZSL task, but it will sacrifice the accuracy of training (seen) classes.