CS 1699: Intro to Computer Vision Matlab Tutorial Prof. Adriana Kovashka University of Pittsburgh September 3, 2015

Plan for Today Course basics refresher Matlab tutorial Overview of Homework 1 (Start on image filtering)

Course Info Course website: Instructor: Adriana Kovashka – Please use "CS1699" at the beginning of your Subject Office: Sennott Square 5325 Office hours: Tuesday and Thursday, 4pm-5pm Grader: Nils Murrugarra-Llerena

Textbooks Computer Vision: Algorithms and Applications by Richard Szeliski Computer Vision: Algorithms and Applications Visual Object Recognition by Kristen Grauman and Bastian Leibe Visual Object Recognition Please do the assigned readings before class

Homework Submission We will use CourseWeb Navigate to the CourseWeb page for CS1699, click on "Assignments" and the corresponding HW # Attach a zip file with your written responses and code Name the file as YourFirstName_YourLastName.zip or YourFirstName_YourLastName.tar Homework is due at 11:59pm on the due Grades will appear on CourseWeb

Late Policy You get 3 "free" late days, i.e., you can submit homework a total of 3 days late. For example, you can submit one problem set 12 hours late, and another 60 hours late. Once you've used up your free late days, you will incur a penalty of 25% from the total project credit possible for each late day. A late day is anything from 1 minute to 24 hours.

Collaboration Policy You will work individually. The work you turn in must be your own work. You can discuss the problem sets with your classmates, but do not look at their code. You cannot use posted solutions, search for code on the internet or use Matlab's implementations of something you are asked to write. When in doubt, ask the instructor! Plagiarism will cause you to fail the class and receive disciplinary penalty.

Homework 1 Part I: Matlab exercise – Today’s lecture Part II: Short answers – Lectures on 9/08 and 9/15 Part III: Implementing image pyramids – Lecture on 9/10 Part IV: Implementing seam carving – Today’s lecture

Matlab tutorial

Image Formation Slide credit: Derek Hoiem

Digital camera A digital camera replaces film with a sensor array Each cell in the array is light-sensitive diode that converts photons to electrons Slide by Steve Seitz

Slide credit: Derek Hoiem Digital images

Sample the 2D space on a regular grid Quantize each sample (round to nearest integer) Image thus represented as a matrix of integer values. Adapted from S. Seitz 2D 1D

Digital color images Slide credit: Kristen Grauman

RG B Color images, RGB color space Digital color images Slide credit: Kristen Grauman

Images in Matlab Images represented as a matrix Suppose we have a NxM RGB image called “im” –im(1,1,1) = top-left pixel value in R-channel –im(y, x, b) = y pixels down, x pixels to right in the b th channel –im(N, M, 3) = bottom-right pixel in B-channel imread(filename) returns a uint8 image (values 0 to 255) –Convert to double format (values 0 to 1) with im2double R G B row column Slide credit: Derek Hoiem

Tutorial Part I s/matlab-tutorial/ s/matlab-tutorial/

Tutorial Part II orial.m orial.m

Homework 1 Seam carving

Seam carving: main idea [Shai & Avidan, SIGGRAPH 2007]

Content-aware resizing Traditional resizing Seam carving: main idea [Shai & Avidan, SIGGRAPH 2007]

Content-aware resizing Seam carving: main idea Intuition: Preserve the most “interesting” content  Prefer to remove pixels with low gradient energy To reduce or increase size in one dimension, remove irregularly shaped “seams”  Optimal solution via dynamic programming. Kristen Grauman, UT-Austin

Want to remove seams where they won’t be very noticeable: –Measure “energy” as gradient magnitude Choose seam based on minimum total energy path across image, subject to 8-connectedness. Seam carving: main idea Kristen Grauman, UT-Austin

Let a vertical seam s consist of h positions that form an 8-connected path. Let the cost of a seam be: Optimal seam minimizes this cost: Compute it efficiently with dynamic programming. Seam carving: algorithm s1s1 s2s2 s3s3 s4s4 s5s5 Kristen Grauman, UT-Austin

How to identify the minimum cost seam? First, consider a greedy approach: Energy matrix (gradient magnitude) Kristen Grauman, UT-Austin

row i-1 Seam carving: algorithm Compute the cumulative minimum energy for all possible connected seams at each entry (i,j): Then, min value in last row of M indicates end of the minimal connected vertical seam. Backtrack up from there, selecting min of 3 above in M. j-1 j row i M matrix: cumulative min energy (for vertical seams) Energy matrix (gradient magnitude) j j+1 Kristen Grauman, UT-Austin

Example Energy matrix (gradient magnitude) M matrix (for vertical seams) Kristen Grauman, UT-Austin

Example Energy matrix (gradient magnitude) M matrix (for vertical seams) Kristen Grauman, UT-Austin

Real image example Original Image Energy Map Blue = low energy Red = high energy Kristen Grauman, UT-Austin

Real image example Kristen Grauman, UT-Austin

Real image example Kristen Grauman, UT-Austin

Other notes on seam carving Analogous procedure for horizontal seams Can also insert seams to increase size of image in either dimension –Duplicate optimal seam, averaged with neighbors Other energy functions may be plugged in –E.g., color-based, interactive,… Can use combination of vertical and horizontal seams Kristen Grauman, UT-Austin

Results from Eunho Yang Example results from classes at UT Austin

Results from Suyog Jain

Seam carving result Original image Results from Martin Becker Conventional resize

Seam carving result Conventional resize Original image Results from Martin Becker

Results from Jay Hennig Original image (599 by 799) Conventional resize (399 by 599) Seam carving (399 by 599)

“Failure cases” with seam carving By Donghyuk Shin

“Failure cases” with seam carving By Suyog Jain

Next Time Image filtering Reading for Tuesday: Szeliski Sec , 3.2 Reading for today was: Szeliski Sec