1 Storage of images for Efficient Retrieval  Representing IDB as relations  straightforward  Representing IDB with spatial data structures  represent the image as rectangles  Representing IDB using image transformations  can be represented by a vector of length k (k is between 100 and 200)  However,  different photographs of the same person may vary, depending on a variety of factors F time of the day at which the photographs were taken F the lighting conditions F the camera used F the exact position of the subject’s head and facial expression F etc.

2 Representing IDB with Relations  (1) Create a relation called images having the scheme:  (Image, ObjID, XLB,XUB,YLB,YUB) F where Image is the name of the image file and (XLB,XUB,YLB,YUB) represent the rectangle in question  Suppose R is a rectangle specified by (XLB, XUB,YLB,YUB) and R is in Rec(I), the there exists a tuple F (I, newid, XLB XUB,YLB,YUB)  (2) For each property p, create a relation R p having the scheme:  (Image, XLB, XUB,YLB,YUB, Value) F where Image is the image file and the (XLB, XUB,YLB,YUB) denote a rectangular cell in the image  Properties can be F Pixel level properties (RGB values) F Object/region level properties (Name, age) F Image level properties (when image was captured, where, by whom)

3 Querying IDB Relations  Eliciting the contents of an image is done using image processing algorithms  Image processing algorithms are only partially accurate  This implies that tuples placed in an IDB relation by an image processing program has certain associated probabilistic attributes  The probability that John Lee is the name of o 2 is 0.75  The probability that Ken Yip is the name of o 2 is 0.15  There is a 10% missing probability ObjId Name Prob o 1 Jim Hatch 0.8 o 2 John Lee 0.75 o 2 Ken Yip 0.15 o 3 John Lee 1

4 Complex Queries  Query: What is the probability that pic1.gif contains both Jim Hatch and Ken Yip? (assume o 1 and o 2 are in pic1.gif)  Is it 0.8*0.15 = 0.12?  In general, the answer is NO ObjId Name Prob o 1 Jim Hatch 0.8 o 2 John Lee 0.75 o 2 Ken Yip 0.15 o 3 John Lee 1

5 Complex Queries  Consider pic8.gif with o 10 and o 11  Four possibilities  possibility 1: o 10 is Ken Yip and o 11 is John Hatch  possibility 2: o 10 is Ken Yip and o 11 is not John Hatch  possibility 3: o 10 is not Ken Yip but o 11 is John Hatch  possibility 4: o 10 is not Ken Yip and o 11 is not John Hatch  Suppose p i denotes the probability of possibility i  p 1 + p 2 = 0.5  p 3 + p 4 = 0.5  p 1 + p 3 = 0.8  p 2 + p 4 = 0.2  p 1 + p 2 + p 3 + p 4 = 1 ObjId Name Prob o 10 Ken Yip 0.5 o 10 Jim Hatch 0.4 o 11 Jim Hatch 0.8 o 11 John Lee 0.1

6 Interval Probability  Need to solve these linear equations to determine the probability that pic8.gif contains both Ken Yip and John Hatch  The result is an interval of probabilities  p 1 between 0.3 and 0.5  Requires the use of Interval probabilities!!!  Interval probabilities allow us to represent the margin of error of image processing algorithms in identifying the object  e.g. with 3% error ObjId Name Prob(l) Prob(u) o 1 Jim Hatch o 2 John Lee o 2 Ken Yip o 3 John Lee

7 Representing image DBs with R-Trees  Create a relation with two attributes (ImageId, ObjId)  Create an R-tree that stores rectangles  if the same rectangle appears in two images, then we have an overflow list  Each rectangle has an associated set of fields that specifies the object/region level properties  Not good for nearest neighbor queries as the tree is constructed with only two dimensions out of the n+2 dimensions of the image  Generalized R-Trees use all the n+2 dimensions to construct the tree

8 Implementations  Many use object-oriented implementation  Support methods such as  rotate, segment, edit  Most implementations assume the whole image to compare  Perform feature extraction and thus represent an image as a vector of n fields  an index is created on an n-dimensional vector  multidimensional extension of point quadtree  R-Tree  To perform similarity search, they compute the Euclidian distance between the vector representing the query image and those of all images