Computational Vision Jitendra Malik, UC Berkeley

Computer Vision Group University of California Berkeley From Pixels to Perception Tiger Grass Water Sand outdoor wildlife Tiger tail eye legs head back shadow mouse

Computer Vision Group University of California Berkeley Object Category Recognition

Computer Vision Group University of California Berkeley Detection can be very fast On a task of judging animal vs no animal, humans can make mostly correct saccades in 150 ms (Kirchner & Thorpe, 2006) –Comparable to synaptic delay in the retina, LGN, V1, V2, V4, IT pathway. –Doesn’t rule out feed back but shows feed forward only is very powerful

EZ-Gimpy Results (Mori & Malik 03) 171 of 192 images correctly identified: 92 % horse smile canvas spade join here

Computer Vision Group University of California Berkeley Caltech-101 [Fei-Fei et al. 04] 102 classes, images/class

Computer Vision Group University of California Berkeley Caltech 101 classification results (By combining cues, one can get above 80% !)

Looking at People 3-pixel man Blob tracking 300-pixel man Limb shape Far fieldNear field

Medium-field The 30-Pixel Man

Examples of Actions Movement and posture change –run, walk, crawl, jump, hop, swim, skate, sit, stand, kneel, lie, dance (various), … Object manipulation –pick, carry, hold, lift, throw, catch, push, pull, write, type, touch, hit, press, stroke, shake, stir, turn, eat, drink, cut, stab, kick, point, drive, bike, insert, extract, juggle, play musical instrument (various)… Conversational gesture –point, … Sign Language

Classifying Ballet Actions 16 Actions. Men used to classify women and vice versa.

What makes computer vision interesting? Great scientific problem –30-50% of the brain is devoted to it –Visual perception has been richly studied –Long history with contributions from greats such as Euclid, Maxwell, Helmholtz, Mach, Schrodinger etc Great engineering problem –Search on the web for images/video –Enhancing visual experiences –Essential for robotics and AI Finally, we are making great progress –Availability of computing resources –Large collections make possible the use of machine learning techniques –Adoption of interdisciplinary approach