1. Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology 1 Modeling Semantic Similarities in Multiple Maps Presenter : Wei-Hao Huang Authors : Laurens van der Maaten, Geoffrey Hinton EWI-ICT TR, 2009

2. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 2 Outlines Motivation Objectives Methodology Experiments Conclusions Comments

3. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 3 Motivation  Semantic space models cannot faithfully represent intransitive pairwise similarities or the similarities of words that have multiple meanings. ─ Triangle inequality ─ Nearest neighbor is limited ─ Similarities are symmetric tie suit tuxedo rope knot Animal dog China North Korea

4. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Objectives 4 To propose multiple map SNE to solve fundamental limitations of metric spaces tie suit tuxedo rope knot tie suit tuxedo tie rope knot

5. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 5 Methodology  Stochastic neighbor embedding  Multiple maps SNE Map Map2 Map1 Map3 Data SNE Multiple maps SNE Data Mixing proportion (importance)

6. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Methodology  Stochastic neighbor embedding 6 tie suit tuxedo rope knot

7. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Methodology  Multiple map SNE 7 tie ropesuit tie animal dog

8. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Methodology  Multiple map SNE 8 A*C=1*1/2=1/2 B*C=1*1/2=1/2

9. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Experiments  Visualization Experiments ─ Florida State University word association dataset ─ Selecting 5019 words  Generalization Experiments ─ To evidence their model for semantic representation ─ Training data: 80% ─ Validation data: 10% ─ Test data: 10% 9

10. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Experiments  Visualization 10

11. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Experiments 11 Sport Clothing Statue of Liberty Cheerleader Tie monarchy

12. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Experiments  Generalization 12

13. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Experiments  Comparing multiple maps SNE with other method. ─ Semantic space models ─ Semantic networks ─ Topic models 13

14. Intelligent Database Systems Lab N.Y.U.S.T. I. M. Conclusions  The multiple maps SNE alleviates the fundamental limitations of metric spaces.  Multiple map model has characteristics that are similar to those of topic models. 14

15. Intelligent Database Systems Lab N.Y.U.S.T. I. M. 15 Comments  Advantages ─ Multiple maps SNE alleviates the fundamental limitations of metric spaces  Applications ─ Data visualization ─ Semantic similarities