Andrew Ng CS228: Deep Learning & Unsupervised Feature Learning Andrew Ng TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA Pieter Abbeel Adam Coates Zico Kolter Ian Goodfellow Quoc Le Honglak Lee Rajat Raina Andrew Saxe

Andrew Ng How is computer perception done? Image Low-level vision features Recognition Image Grasp point Low-level features Object detection Computer vision is hard!

Andrew Ng How is computer perception done? Image Grasp point Low-level features Image Vision features Recognition Object detection Audio Audio features Speaker ID Audio classification NLP Text Text features Text classification, MT, IR, etc.

Andrew Ng Sensor representations Input Learning/AI algorithm Low-level features

Andrew Ng Computer vision features SIFT Spin image HoG RIFT Textons GLOH

Andrew Ng Audio features ZCR Spectrogram MFCC Rolloff Flux

Andrew Ng NLP features Parser features NER/SRL Stemming POS tagging Anaphora WordNet features

Andrew Ng A plethora of sensors Camera array 3d range scan (laser scanner) 3d range scans (flash lidar) Audio A general-purpose algorithm for good sensor representations? Visible light image Thermal Infrared

Andrew Ng Sensor representation in the brain [BrainPort; Martinez et al; Roe et al.] Seeing with your tongue Human echolocation (sonar) Auditory cortex learns to see. Auditory Cortex

Andrew Ng Learning abstract representations pixels edges object parts (combination of edges) object models [Related work: Deep learning, Hinton, Bengio, LeCun, and others.]

Andrew Ng Feature learning for audio Learned features correspond to phonemes and other “basic units” of sound. Learned features Algorithm :

Andrew Ng TIMIT Phone classificationAccuracy Prior art (Clarkson et al.,1999) 79.6% Stanford Feature learning 80.3% TIMIT Speaker identificationAccuracy Prior art (Reynolds, 1995) 99.7% Stanford Feature learning 100.0% Audio Images Multimodal (audio/video) CIFAR Object classificationAccuracy Prior art (Yu and Zhang, 2010) 74.5% Stanford Feature learning 79.6% NORB Object classificationAccuracy Prior art (Ranzato et al., 2009) 94.4% Stanford Feature learning 97.0% AVLetters Lip readingAccuracy Prior art (Zhao et al., 2009) 58.9% Stanford Feature learning 65.8% Galaxy Other feature learning records: Different phone recognition task (Hinton), PASCAL VOC object classification (Yu) Hollywood2 ClassificationAccuracy Prior art (Laptev et al., 2004) 48% Stanford Feature learning 53% KTHAccuracy Prior art (Wang et al., 2010) 92.1% Stanford Feature learning 93.9% UCFAccuracy Prior art (Wang et al., 2010) 85.6% Stanford Feature learning 86.5% YouTubeAccuracy Prior art (Liu et al., 2009) 71.2% Stanford Feature learning 75.8% Video