Stockman MSU/CSE Fall 2009 Computer Vision: CSE 803 A brief intro.

Slides:



Advertisements
Similar presentations
Human-Computer Interaction
Advertisements

November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.
Digital Image Fundamentals Selim Aksoy Department of Computer Engineering Bilkent University
Optics and Human Vision The physics of light
Computational Photography: Color perception, light spectra, contrast Connelly Barnes.
COLORCOLOR A SET OF CODES GENERATED BY THE BRAİN How do you quantify? How do you use?
Color Image Processing
Stockman MSU/CSE Fall Computer Vision and Human Perception A brief intro from an AI perspective.
Computer Vision: CSE 803 A brief intro MSU/CSE Fall 2014.
MSU/CSE Fall Computer Vision: Imaging Devices Summary of several common imaging devices covered ( Chapter 2 of Shapiro and Stockman)
Digital Image Processing: Digital Imaging Fundamentals.
CSE 803: image compression Why? How much? How? Related benefits and liabilities.
Lecture 30: Light, color, and reflectance CS4670: Computer Vision Noah Snavely.
Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009.
MSU CSE 803 Stockman Fall 2009 Vectors [and more on masks] Vector space theory applies directly to several image processing/representation problems.
1 Imaging and Image Representation  Sensing Process  Typical Sensing Devices  Problems with Digital Images  Image Formats  Relationship of 3D Scenes.
1 Computer Vision: Week 1 Intro. What are its goals? What are the applications What are some ways of using images (Later: methods and programming) MSU/CSE.
Stockman MSU/CSE Fall Computer Vision: Week 1 Intro. What are its goals? What are the applications What are some ways of using images (Later: methods.
1 Color Color Used heavily in human vision Used heavily in human vision Color is a pixel property, making some recognition problems easy Color is a pixel.
Stockman MSU/CSE Fall Computer Vision: Imaging Devices Summary of several common imaging devices covered in Chapter 2 of Shapiro and Stockman.
Color: Readings: Ch 6: color spaces color histograms color segmentation.
Chapter 2 Computer Imaging Systems. Content Computer Imaging Systems.
Visual Cognition I basic processes. What is perception good for? We often receive incomplete information through our senses. Information can be highly.
Stockman MSU Fall Computing Motion from Images Chapter 9 of S&S plus otherwork.
Ch 31 Sensation & Perception Ch. 3: Vision © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics –convergence –Inhibition, lateral.
CSE 803 Stockman Fall Color Used heavily in human vision Color is a pixel property, making some recognition problems easy Visible spectrum for humans.
Stockman MSU/CSE Fall Computer Vision: Week 1 Intro. What are its goals? What are the applications What are some ways of using images (Later: methods.
1B50 – Percepts and Concepts Daniel J Hulme. Outline Cognitive Vision –Why do we want computers to see? –Why can’t computers see? –Introducing percepts.
Human Visual System 4c8 Handout 2. Image and Video Compression We have already seen the need for compression We can use what we have learned in information.
ECE 472/572 – Digital Image Processing Lecture 2 – Elements of Visual Perception and Image Formation 08/25/11.
A Brief Overview of Computer Vision Jinxiang Chai.
SCCS 4761 Introduction What is Image Processing? Fundamental of Image Processing.
Visual Perception How We See Things. Visual Perception It is generally agreed that we have five senses o Vision o Hearing o Touch o Taste o Smell Of our.
Visual Perception How We See Things.
Perception Introduction Pattern Recognition Image Formation
Visual Perception PhD Program in Information Technologies Description: Obtention of 3D Information. Study of the problem of triangulation, camera calibration.
Computer Vision – Fundamentals of Human Vision Hanyang University Jong-Il Park.
Human Visual Perception The Human Eye Diameter: 20 mm 3 membranes enclose the eye –Cornea & sclera –Choroid –Retina.
Digital Image Fundamentals. What Makes a good image? Cameras (resolution, focus, aperture), Distance from object (field of view), Illumination (intensity.
Image Formation CS418 Computer Graphics John C. Hart.
Ch 31 Sensation & Perception Ch. 3: Vision © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics –convergence –Inhibition, lateral.
Digital Image Processing & Analysis Dr. Samir H. Abdul-Jauwad Electrical Engineering Department King Fahd University of Petroleum & Minerals.
Introduction to Computer Vision Sebastian van Delden USC Upstate
Chapter 6 Section 2: Vision. What we See Stimulus is light –Visible light comes from sun, stars, light bulbs, & is reflected off objects –Travels in the.
Intelligent Vision Systems Image Geometry and Acquisition ENT 496 Ms. HEMA C.R. Lecture 2.
Tutorial Visual Perception Towards Computer Vision
University of Kurdistan Digital Image Processing (DIP) Lecturer: Kaveh Mollazade, Ph.D. Department of Biosystems Engineering, Faculty of Agriculture,
Lecture 34: Light, color, and reflectance CS4670 / 5670: Computer Vision Noah Snavely.
Image Perception ‘Let there be light! ‘. “Let there be light”
COMPUTER GRAPHICS CS 482 – FALL 2015 SEPTEMBER 1, 2015 HUMAN VISUAL PERCEPTION EYE PHYSIOLOGY COLOR BLINDNESS CONSTANCY SHADOWS PARALLAX STEREOSCOPY.
Introduction to Image Processing. What is Image Processing? Manipulation of digital images by computer. Image processing focuses on two major tasks: –Improvement.
Intelligent Vision Systems Image Geometry and Acquisition ENT 496 Ms. HEMA C.R. Lecture 2.
1 Review and Summary We have covered a LOT of material, spending more time and more detail on 2D image segmentation and analysis, but hopefully giving.
Image Perception ‘Let there be light! ‘. “Let there be light”
COMP 9517 Computer Vision Digital Images 1/28/2018 COMP 9517 S2, 2009.
Vision.
CS262 – Computer Vision Lect 4 - Image Formation
- photometric aspects of image formation gray level images
Color Image Processing
Color Image Processing
EE663-Digital Image Processing & Analysis Dr. Samir H
Digital Image Processing (DIP)
Color Image Processing
Vision.
Color Image Processing
Color Image Processing
Introduction Computer vision is the analysis of digital images
Experiencing the World
Presentation transcript:

Stockman MSU/CSE Fall 2009 Computer Vision: CSE 803 A brief intro

Stockman MSU/CSE Fall 2009 Computer Vision What are the goals of CV? What are the applications? How do humans perceive the 3D world via images? Some methods of processing images. What are the major research areas?

Stockman MSU/CSE Fall 2009 Goal of computer vision Make useful decisions about real physical objects and scenes based on sensed images. Alternative (Aloimonos and Rosenfeld): goal is the construction of scene descriptions from images. How do you find the door to leave? How do you determine if a person is friendly or hostile?.. an elder?.. a possible mate?

Stockman MSU/CSE Fall 2009 Critical Issues Sensing: how do sensors obtain images of the world? Information/features: how do we obtain color, texture, shape, motion, etc.? Representations: what representations should/does a computer [or brain] use? Algorithms: what algorithms process image information and construct scene descriptions?

Stockman MSU/CSE Fall 2009 Root and soil next to glass

Stockman MSU/CSE Fall 2009 Images: 2D projections of 3D 3D world has color, texture, surfaces, volumes, light sources, objects, motion, betweeness, adjacency, connections, etc. 2D image is a projection of a scene from a specific viewpoint; many 3D features are captured, some are not. Brightness or color = g(x,y) or f(row, column) for a certain instant of time Images indicate familiar people, moving objects or animals, health of people or machines

Stockman MSU/CSE Fall 2009 Image receives reflections Light reaches surfaces in 3D Surfaces reflect Sensor element receives light energy Intensity matters Angles matter Material maters

Stockman MSU/CSE Fall 2009 Simple objects: simple image?

Stockman MSU/CSE Fall 2009 Where is the sun?

Stockman MSU/CSE Fall 2009 CCD Camera has discrete elts Lens collects light rays CCD elts replace chemicals of film Number of elts less than with film (so far)

Stockman MSU/CSE Fall 2009 Camera + Programs = Display Camera inputs to frame buffer Program can interpret data Program can add graphics Program can add imagery

Stockman MSU/CSE Fall 2009 Some image format issues Spatial resolution; intensity resolution; image file format

Stockman MSU/CSE Fall 2009 Resolution is “pixels per unit of length” Resolution decreases by one half in cases at left Human faces can be recognized at 64 x 64 pixels per face

Stockman MSU/CSE Fall 2009 Features detected depend on the resolution Can tell hearts from diamonds Can tell face value Generally need 2 pixels across line or small region (such as eye)

Stockman MSU/CSE Fall 2009 Human eye as a spherical camera 100M sensing elts in retina Rods sense intensity Cones sense color Fovea has tightly packed elts, more cones Periphery has more rods Focal length is about 20mm Pupil/iris controls light entry Eye scans, or saccades to image details on fovea 100M sensing cells funnel to 1M optic nerve connections to the brain

Stockman MSU/CSE Fall 2009 Look at some CV applications Graphics or image retrieval systems; Geographical: GIS; Medical image analysis; manufacturing

Stockman MSU/CSE Fall 2009 Aerial images & GIS Aerial image of Wenatchie River watershed Can correspond to map; can inventory snow coverage

Stockman MSU/CSE Fall 2009 Medical imaging is critical Visible human project at NLM Atlas for comparison Testbed for methods

Stockman MSU/CSE Fall 2009 Manufacturing case 100 % inspection needed Quality demanded by major buyer Assembly line updated for visual inspection well before today’s powerful computers

Stockman MSU/CSE Fall 2009 Simple Hole Counting Alg. Customer needs 100% inspection About 100 holes Big problem if any hole missing Implementation in the 70’s Alg also good for counting objects See auxiliary slides

Stockman MSU/CSE Fall 2009 Some hot new applications Phototourism: from hundreds of overlapping images, maybe some from cell phones, construct a 3D textured model of the landmark[s] Photo-GPS: From a few cell phone images “the web” tells you where you are located [perhaps using the data as above]

Stockman MSU/CSE Fall 2009 Image processing operations Thresholding; Edge detection; Motion field computation

Stockman MSU/CSE Fall 2009 Find regions via thresholding Region has brighter or darker or redder color, etc. If pixel > threshold then pixel = 1 else pixel = 0

Stockman MSU/CSE Fall 2009 Example red blood cell image Many blood cells are separate objects Many touch – bad! Salt and pepper noise from thresholding How useable is this data?

Stockman MSU/CSE Fall 2009 sign = imread('Images/stopSign.jpg','jpg'); red = (sign(:, :, 1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'Images/stopRed120.jpg', 'jpg')

Stockman MSU/CSE Fall 2009 sign = imread('Images/stopSign.jpg','jpg'); red = (sign(:, :, 1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'Images/stopRed120.jpg', 'jpg')

Stockman MSU/CSE Fall 2009 Thresholding is usually not trivial

Stockman MSU/CSE Fall 2009 Can cluster pixels by color similarity and by adjacency Original RGB ImageColor Clusters by K-Means

Stockman MSU/CSE Fall 2009 Detect Motion via Subtraction Constant background Moving object Produces pixel differences at boundary Reveals moving object and its shape Differences computed over time rather than over space

Stockman MSU/CSE Fall 2009 Two frames of aerial imagery Video frame N and N+1 shows slight movement: most pixels are same, just in different locations.

Stockman MSU/CSE Fall 2009 Best matching blocks between video frames N+1 to N (motion vectors) The bulk of the vectors show the true motion of the airplane taking the pictures. The long vectors are incorrect motion vectors, but they do work well for compression of image I2! Best matches from 2 nd to first image shown as vectors overlaid on the 2 nd image. (Work by Dina Eldin.)

Stockman MSU/CSE Fall 2009 Gradient from 3x3 neighborhood Estimate both magnitude and direction of the edge.

Stockman MSU/CSE Fall rows of intensity vs difference

Stockman MSU/CSE Fall 2009 Boundaries not always found well

Stockman MSU/CSE Fall 2009 Canny edge operator

Stockman MSU/CSE Fall 2009 Mach band effect shows human bias Biology consistent with image processing operations

Stockman MSU/CSE Fall 2009 Human bias and illusions supports receptive field theory of edge detection

Stockman MSU/CSE Fall 2009 Color and shading Used heavily in human vision Color is a pixel property, making some recognition problems easy Visible spectrum for humans is 400nm (blue) to 700 nm (red) Machines can “see” much more; ex. X-rays, infrared, radio waves

Stockman MSU/CSE Fall 2009 Imaging Process (review)

Stockman MSU/CSE Fall 2009 Factors that Affect Perception Light: the spectrum of energy that illuminates the object surface Reflectance: ratio of reflected light to incoming light Specularity: highly specular (shiny) vs. matte surface Distance: distance to the light source Angle: angle between surface normal and light source Sensitivity how sensitive is the sensor

Stockman MSU/CSE Fall 2009 CV: Perceiving 3D from 2D Many cues from 2D images enable interpretation of the structure of the 3D world producing them

Stockman MSU/CSE Fall 2009 Many 3D cues How can humans and other machines reconstruct the 3D nature of a scene from 2D images? What other world knowledge needs to be added in the process?

Stockman MSU/CSE Fall 2009 What about models for recognition “recognition” = to know again; How does memory store models of faces, rooms, chairs, etc.?

Stockman MSU/CSE Fall 2009 Some methods: recognize Via geometric alignment: CAD Via trained neural net Via parts of objects and how they join Via the function/behavior of an object

Stockman MSU/CSE Fall 2009 summary Images have many low level features Can detect uniform regions and contrast Can organize regions and boundaries Human vision uses several simultaneous channels: color, edge, motion Use of models/knowledge diverse and difficult Last 2 issues difficult in computer vision