1. Laso: Label-Set Operations Networks for Multi-label Few-shot Learning
CVPR 2019 oral
Laso: Label-Set Operations Networks for Multi-label Few-shot Learning
Amit Alfassy, Leonid Karlinsky, Amit Aides
IBM Research AI
Haifa, Israel
Fewshot: Only a handful of examples are available for the task

2. A story: The Tank Recognition Project
In 1980, the America government started a project for tank recognition.

3. A story: The Tank Recognition Project
In 1980, the America government started a project for tank recognition.
“0”
“1”

4. A story: The Tank Recognition Project
In 1980, the America government started a project for tank recognition.
“0”
“1”
The accuracy on the test dataset was high,
but when it turned into real world scenarios, it did not work at all !

5. A story: The Tank Recognition Project
In 1980, the America government started a project for tank recognition.
Why?
“0”
“1”
The accuracy on the test dataset was high,
but when it turned into real world scenarios, it did not work at all !

6. A story: The Tank Recognition Project
In 1980, the America government started a project for tank recognition.
“0”
“1”
Sunny
Rainy

7. A story: The Tank Recognition Project
Machine Learning Researcher: It’s overfitting!
Apply data augmentation、collect more images from different weather condition

8. A story: The Tank Recognition Project
Researcher of Interpretability of NN: The recognition process is a black-box!
Visualize the locations that fire on the recognition task

9. A story: The Tank Recognition Project
After I read this paper: The tank is coupled with the weather!
De-couple them from each other!

10. A story: The Tank Recognition Project
After I read this paper: The tank is coupled with the weather!
De-couple them from each other!
In multi-label few-shot learning, objects may easily coupled from each other, resulting in undesirable results.

11. Methodology

12. Methodology 𝒁 𝒊𝒏𝒕 = 𝑴 𝒊𝒏𝒕 ( 𝑭 𝒙 , 𝑭 𝒚 ) 𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃 ( 𝑭 𝒙 , 𝑭 𝒚 )

13. Methodology 𝐶 𝑙𝑜𝑠𝑠 𝒁 𝒊𝒏𝒕 = 𝑴 𝒊𝒏𝒕 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝐶 𝑙𝑜𝑠𝑠

14. Methodology 𝐿𝑎𝑠𝑜 𝑙𝑜𝑠𝑠 𝒁 𝒊𝒏𝒕 = 𝑴 𝒊𝒏𝒕 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝐿𝑎𝑠𝑜 𝑙𝑜𝑠𝑠

15. Methodology 𝑅 𝑙𝑜𝑠𝑠 𝑠𝑦𝑚 𝒁 𝒊𝒏𝒕 = 𝑴 𝒊𝒏𝒕 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝑅 𝑙𝑜𝑠𝑠 𝑠𝑦𝑚

16. Methodology 𝑅 𝑙𝑜𝑠𝑠 𝑚𝑐 𝒁 𝒊𝒏𝒕 = 𝑴 𝒊𝒏𝒕 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃 ( 𝑭 𝒙 , 𝑭 𝒚 )
𝑅 𝑙𝑜𝑠𝑠 𝑚𝑐

17. Experiments Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset(40 facial attributes)

18. Experiments Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset (40 facial attributes)
The Coco dataset is split into Coco-A(64 classes), Coco-B(16 classes)
𝐹 𝐴 , 𝑙𝑎𝑠𝑜 𝐴 , 𝐶 𝐴
𝐹 𝐴 , 𝑙𝑎𝑠𝑜 𝐴 , 𝐶 𝐵
Numbers are in mAP
Numbers are in mIoU

19. Experiments Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset (40 facial attributes)

20. Experiments: Ablations
Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset (40 facial attributes)
Tab. Analytic approximations to set operations

21. Experiments: Ablations
Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset (40 facial attributes)
Tab. Analytic approximations to set operations

22. Experiments: Ablations
Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset (40 facial attributes)
Tab. Serve as data augmentation strategy
numbers are in mAP
Conduct on Coco-B

23. Conclusion
+ First paper to address the multi-label classification of few-shot learning.
+ The Motivation —— which I would call it “de-couple” is novel.
+ Abundant experiments, well written and easy to follow.
- The implementation of de-couple is disappointing.