1. CORNELL UNIVERSITY CS 764 Seminar in Computer Vision Ramin Zabih Fall 1998

2. CORNELL UNIVERSITY 2 Course mechanics n Meeting time will be Tue/Thu 11-12, here Starting a week from today n Home page is now up www/CS764 n Assignment: present one paper You’ll have a lot of freedom, but you need to talk to me in advance Some possible papers will be posted shortly

3. CORNELL UNIVERSITY 3 Topic of this seminar n The use of “knowledge” in the analysis of visual data Sometimes called “context” n Clearly this is vital On both psychological and technical grounds But how? No one has much of an idea… n What is the interface between reasoning and perception? (Or, mind and body?)

4. CORNELL UNIVERSITY 4 What is the visual system’s “contract” n Two standard (bad) answers n Answer 1: describe the scene in terms of surfaces [low-level vision] There is a green patch 2” wide 1’ away n Answer 2: describe the scene in terms of objects [model-based recognition] Start with a set of 3D models (modelbase) Determine position and pose

5. CORNELL UNIVERSITY 5 Why are these answers wrong? n They are almost purely data-driven Bottom-up (from the data) versus top-down (from somewhere else) n They report “objective fact”, with no room for the task at hand For a given image, there is only one right answer n Other problems as well Not very useful, etc.

6. CORNELL UNIVERSITY 6 Technical and psychological arguments n There are technical arguments against this Vision is an inverse problem –Many 3D scenes could explain a single 2D image On engineering grounds, this makes no sense –Ultimately, perception is used for some task n The human perceptual system has both top- down and bottom-up elements Various optical illusions –Two people can look at the same picture and see something completely different

10. CORNELL UNIVERSITY 10 Your vision system doesn’t listen

11. CORNELL UNIVERSITY 11 It makes “reasonable” assumptions

12. CORNELL UNIVERSITY 12 Low-level vision has its solution n Inverse problems require assumptions n The assumptions for low-level vision are extremely general (I.e., weak) Reflect the physics of the visible world For example, motion or depth or intensity tend to be “coherent” –Saying that every pixel is moving differently from its neighbors is a very unlikely answer –The world we live in tends not to do that –Helmholtz’s “unconscious inference”

13. CORNELL UNIVERSITY 13 We’ll need high-level vision n Most of the field is low-level vision or model- based recognition Partly to avoid the confusion CS764 is about n Key question: how to avoid brittleness? Can make the visual system compute just what we need for our task (I.e., berries) But how to handle the unexpected (I.e., lions)?

14. CORNELL UNIVERSITY 14 A short historical perspective n 1960’s vision was completely task-specific A black blob in the center of the image is a telephone These efforts are now considered “hacks” n 1970’s vision became completely general Marr pushed the field towards precise technical questions Low-level vision and recognition became dominant

15. CORNELL UNIVERSITY 15 Tasks strike back n In the mid-1980’s, several attempts were made to re-introduce a notion of task Active/animate/purposive vision n These attempts are widely viewed as failures, for good reasons We’ll look at them a bit next week n It’s not enough to have good intuitions There needs to be technical merit as well

16. CORNELL UNIVERSITY 16 Desiderata n Technical solutions (algorithms) that are very roughly consistent with human data Goal is not AI, psychology or philosophy n Provide visual summaries useful for tasks, but degrade gracefully Handle open/unstructured environments Deal with expectations and breakdown

17. CORNELL UNIVERSITY 17 Our path for 764 n No good computational work to read Perhaps Vera will fix this? n We will examine papers along these lines: Computational approaches that failed Psychological data that is highly suggestive Neurologically inspired architectures Cognitive scientists and philosophers –Their goal is argument, not algorithm! –They’ve thought the most about these issues